I've soldered all the components plus one row of LEDs so far, but nothing lights up when powered from a power adapter.

The power supply is reading 5.3V when not powering the Peggy, but I'm only getting 0.1V between GND and Vcc when it's plugged in.

I'm going to try with batteries instead, just off out to buy some... What other info would be useful for debugging?

It sounds like you have a short circuit on your kit between power and ground. The power supply is shutting down-- going to low voltage-- because of the short circuit.

You can probably verify this by unplugging it and checking the resistance with a multimeter.

Check the chip orientation and check the board carefully for leads on the back that are touching when they shouldn't be, as well as for stray scraps of metal.

Yes, was a dodgy solder join around the power adapter. 'Tis working great now. Thanks for the quick response!