I try very hard to buy fresh, local food. I visit farm stands, farmer's market and craft shows (a great place to find baked goods, jerky, jam, etc). When the things I'm looking for are not available in those locales, or if those locales are unavailable (Michigan winters don't keep many farm stands in business!), I have to get them from grocery stores. I often shop at the local VG's chain and sometimes hit the (also local) Hiller's. I also hit the Michigan-based Meijer on occasion. All of these chains carry some organic products and they even mark the Michigan-based brands. If your local market doesn't carry many local items, talk to them and see if they'll start. Also ask them to mark the local products to build awareness.
Now, all that is well and good, but I must admit to a weakness. Costco. Buying in bulk, when done correctly (only buying what you'll use, comparing unit prices, etc) saves a ton of time and money. Not to mention packaging. Imagine my happy surprise when I began finding wonderful organic products in their vast aisles! Just goes to show that you can't discount any options when looking for affordable organic options.