Take the offending thing off. Put a large flat washer on the lever, bend another one to make it into a spring washer and place it on top, grease the bushing and put the part with the rubber back on, another washer on top and clean the threads and use Locktite to put the nut on rather tight. If you have used the correct thickness of washers and the one with the bend gives a good bit of tension...the pedal will stay put, and you just have to once in a while re-tighten it or remove and re-bend the spring washer. The rubber can be kept on too if you have the typical problem of it flying off when you least want it to...at the end, drill a hole, put a large washer over the end of the thing and use a cotter pin to hold the washer on...no more barked ankles..... :wink:
Others will have maybe neater solutions...wait and see what they suggest and make your choice.....