I tried putting a Hubsan h107l flight controller into a Syma x5 crash pack (frame, motors, etc).

Everything is hooked up correctly, but the copter is unable to stabilize, glopping wildly as you try to take off. I have learned it is PID programming which stabilizes the craft. Perhaps PID settings are very different for shaft mount propeller micro-quads from geared propeller x5 size quads.

I suppose my experiment is a success, in determining something that wont work,, but I am hoping to learn more and of course result in a working quad, with the more aggressive tilt and yaw, but still a humble, safe, cheap feather-weight x5 clone.

Is there anything I could do to make it work?

You are quite correct that the PIDs which would be appropriate for a tiny little Hubsan H107 won't be appropriate for a much larger craft. Using the H107 flight controller (which is designed to run 6x15mm motors) to run the larger motors on the Syma is risky as the motor drive circuits may not be beefy enough to supply the higher current demands of the larger motors.

If you want to experiment further, you need a flight controller that runs firmware that you can configure yourself.

Have a little Google around for "brushed flight controller".

There's also the excellent 'Silverware' firmware that runs on a variety of toy quad flight controllers. See this thread over on RCGroups.