How to define a map with id and list of records of an object

Yes it seems alright to do it like you have, To add for the first time

mapProductEntity.put(et.Product__c, new List <Entitlement__c> { et });

Even where you're adding it above you don't need to reconstruct Entitlement__c as that is already your loop variable.

So even above you can use

if (mapProductEntity.containsKey(et.Product__c))
            {
mapProductEntity.get(et.Product__c). add(et);

One can use Map.containsKey() to check and initialise a list as indicated, or alternate way could be like this [matter of taste ;)]

Map<id,List<Entity__c>> mapProductEntity = new Map<id,List<Entity__c>>();
for(Entity__c et : [SELECT name,Field1__c, Product__c, Field2__c FROM Entity__c WHERE Product__c IN : Items ORDER BY Product__c ]) {
    List<Entity__c> entitiesForKey = mapProductEntity.get(et.Product__c);
    if (entitiesForKey == null) {
        entitiesForKey = new List<Entity__c>();
        mapProductEntity.put(et.Product__c, entitiesForKey);
    }    
    entitiesForKey.add(new Entity__c( field__c = et.field1__c,product__c = et.product__c, field2__c = et.value__c ));
}

Depending on how your relationships are structured, I would suggest using a nested SOQL statement - so something like:

Map<id,List<Entity__c>> mapProductEntity = new Map<id,List<Entity__c>>();

for (Product2 p: [SELECT Id, Name, (Select Name,Field1__c, Field2__c FROM Entity__r) FROM Product2 WHERE ID IN : Items ORDER BY Name ]) {
mapProductEntity.put(p.Id, p.Entity__r);   
}