Pandas combining sparse columns in dataframe

You can use df.stack() assuming 'id' is your index else set 'id' as index. Then use pd.pivot_table.

df = df.stack().reset_index(name='val',level=1)
df['group'] = 'g'+ df['level_1'].str.extract('col(\d+)')
df['level_1'] = df['level_1'].str.replace('col(\d+)','')
df.pivot_table(index=['id','group'],columns='level_1',values='val')

level_1    cola  colb
id group
1  g1      11.0  12.0
2  g2      21.0  86.0
3  g1      22.0  87.0
4  g3     545.0  32.0

Another alternative with pd.wide_to_long

m = pd.wide_to_long(df,['col'],'id','j',suffix='\d+\w+').reset_index()

(m.join(pd.DataFrame(m.pop('j').agg(list).tolist()))
  .assign(group=lambda x:x[0].radd('g'))
  .set_index(['id','group',1])['col'].unstack().dropna()
  .rename_axis(None,axis=1).add_prefix('col').reset_index())

---

   id group cola colb
0   1    g1   11   12
1   2    g2   21   86
2   3    g1   22   87
3   4    g3  545   32

Use:

import re

def fx(s):
    s = s.dropna()
    group = 'g' + re.search(r'\d+', s.index[0])[0]
    return pd.Series([group] + s.tolist(), index=['group', 'cola', 'colb'])

df1 = df.set_index('id').agg(fx, axis=1).reset_index()

---

# print(df1)

   id   group  cola   colb
0   1    g1    11.0   12.0
1   2    g2    21.0   86.0
2   3    g1    22.0   87.0
3   4    g3    545.0  32.0