Avoid fruit with stickers and obviously fruit packaged in styrofoam and plastic.
Another option is to sign up for an organic vege box delivery scheme. Organic Pathways will help you find a scheme for your area (last time I checked Christchurch, Wellington, Hamilton and Auckland were covered). When we lived in Christchurch we used Just Organic, which was awesome, and we can highly recommend.
If fresh produce isn’t practical try bottled or tinned as most frozen and processed food has plastic packaging.
Another option is to have a go at growing your own!