How to make Advanced Activations Layers in Mathematica?

The documentation of ElementwiseLayer explicitly lists which functions are allowed, and UnitStep is not in this list, I believe that's why the function fails.

Simple workaround: Use a combination of functions from that list, like Ramp:

f = Ramp[#] - Ramp[-#]*0.3 &;

l = ElementwiseLayer[f]
l[{-1, -0.5, 0, 0.5, 1}]

{-0.3, -0.15, 0., 0.5, 1.}

This is how to constract a PReLU

data = Thread[RandomReal[1, {100, 2}] -> RandomReal[1, {100, 3}]];
net = NetGraph[{5, ConstantArrayLayer["Output" -> 1], 
         ReplicateLayer[5], FlattenLayer[], 
         ThreadingLayer[Ramp[#1] - #2*Ramp[-#1] &], 3}, 
         {NetPort["Input"] -> 1 -> 5 -> 6, 2 -> 3 -> 4 -> 5}]

NetTrain[net, data, MaxTrainingRounds -> 100, BatchSize -> 32];
(*well done*)

In 11.2,mma provide more Advanced Activations Layers