Filtering DataFrame on groups where count of element is different than 1

Use series.eq to check if brand is equal to X , then groupby and transform sum and filter groups in which X count is equal to 1:

df[df['brand'].eq('X').groupby(df['group']).transform('sum').eq(1)]

   group brand
0      1     A
1      1     B
2      1     X
7      3     E
8      3     F
9      3     X

Groupby column and apply a simple filter of count of 'X' character in the group equal to 1

df.groupby('group').filter(lambda x: x['brand'].str.count('X').sum() == 1)

Output

   group brand
0      1     A
1      1     B
2      1     X
7      3     E
8      3     F
9      3     X

This should work as well

df[df.groupby(['group'])['brand'].transform('sum').str.count('X').eq(1)]

Output

 group  brand
0   1   A
1   1   B
2   1   X
7   3   E
8   3   F
9   3   X