adding items to a list in a dictionary

Get rid of the tempList and replace your else clause with:

testList.Add(key[index], new List<long> { val[index] });

And don't use Contains. TryGetValue is much better:

for (int index = 0; index < 5; index++)
{
    int k = key[index];
    int v = val[index];
    List<long> items;
    if (testList.TryGetValue(k, out items))
    {
        items.Add(v);
    }
    else
    {
        testList.Add(k, new List<long> { v });
    }
}

You are using the same list for both keys in the Dictionary

    for (int index = 0; index < 5; index++)
    {
        if (testList.ContainsKey(key[index]))
        {
            testList[k].Add(val[index]);
        }
        else
        {
            testList.Add(key[index], new List<long>{val[index]});
        }
    }

Just create one new List(Of Long) when the key doesn't exists then add the long value to it

Replace else with:

else
{
    tempList.Clear();
    tempList.Add(val[index]);
    testList.Add(key[index], new List<long>(tempList));
}

The problem is, you are adding a reference to TempList to both keys, it is the same reference so it gets replaced in the first one.

I am creating a new list so it doesn't get replaced: new List<long>(tempList)