Go to your shipping methods and select "Per Product" as shipment method.
If you want to charge first item let's say $4.99 and each additional item $2. Follow the procedure:
Setup calculation and call it "Handling fee as $2.99. Do it "after tax"