What follows is a poorly formatted write up of the process of trying to get product information out of New World. I succeeded! A lot of it is trying to simplify the process. You don’t need to understand it. The end goal, is to scrape grocery store sites to list products by a custom filter – e.g. cheapest protein.
Intro
It’s hard to kick the habit that I don’t want the cheapest, I want the better quality. I do want the cheapest. But sometimes the cheapest isn’t actually the cheapest. Supermarket websites allow you to sort products by cheapest-per. Cheapest per kilo, cheapest per can. What about cheapest per energy? Cheapest per protein? Cheapest per overall nutritional score?
It’s easy to think that they don’t do this because it would be too hard. Well. Let’s find out. In this initial intro, I’m going to look – manually – at New World. If it proves to be successful, then I’ll try to make it more generic, and eventually make something anyone can use.
But first – how does the internet work?
The internet
This isn’t a course on web programming, and I don’t think I could adequately do one that isn’t better covered by someone else’s YouTube video, but a very brief explainer:
Imagine that you want to send a letter to someone, asking for some information. You need:
Their address
A way of asking them to look something up
A way of telling them what to look up
A way of telling them how you want that information
A way of them sending it back to you
Intuitively, this isn’t hard (although I doubt many people have sent letters now). The address – well that’s easy:
Mr H. Potter The Cupboard under the Stairs 4 Privet Drive Little Whinging Surrey
A format easily understood. You know who you’re sending to, and where they’re going to be. In the internet, this is a URL1 – the thing you type to go to google.com or, in this case, https://www.newworld.co.nz/
That’s the address part. Let’s jump to #5 – a return address. Well, either in your letter you put a return address on the top right, or you include a SAE – Stamped & Addressed Envelope that they can just pop the response in and send back. For now, that’s a close enough analogy2
Now #2-4. Telling someone to look something up, what to look up, and what format you want it in. To a native speaker this is really easy:
“Please could you give me a list of all your flours, and their costs, and send them back in an ordered list?”
In English, we have a fixed set of words that describe “Please give me”. The internet is no different! The key words for today are:
GET (please give me)
POST (take this and remember it)
PUT (take this update and remember it in place of a previous one)
In this case, we want to GET. Then we say what we want to GET – generally people only have a fixed list of things they can GET. After all, you wouldn’t expect a flour shop to be able to give you a list of all their washing machines – at least, not a useful list.
This is called a route, and you can see it in your web browser again – the path broken by ‘/’, for example, if I go to New World and look for bananas I get:
The route is /shop/category/fruit-and-vegetables/fruit/bananas
After that, we can add extra information, like filtering out certain items, with a ‘?’ and a series of ‘thing’=’value’ pairs. Here that extra information is that we’re asking for page 1 only.
What about the format to send it back? Behind the scenes, your web browser is including the information that it would like the content back in, for example, HTML – the language that web browsers understand, or JSON – a convenient format for computers in general to understand.
Seeing it in practice
There’s a lot of information here, and it’s quite small, but on the right we can see a browser making a GET, to http://www.newworld.co.nz, to a path (filename) that ultimately ends up as /category/fruit-and-vegetables/fruit/bananas. You can try this yourself by opening up your console (usually f12) and moving around websites.
But this is pretty inconvenient – it still involves clicking. I want to know if we can do this automatically. When websites and services have a way of interacting with them, they have something called an “API” – a list of all the ways you can interact with them and what words to use. New World have one, but they don’t release it publicly. Fortunately, because they have a website, it shouldn’t be too hard to figure out. In fact, right here we can see a request to api-prod.newworld.co.nz:
Let’s try poking around. For this, I’m going to use Postman – a tool for interacting with APIs
I wanted to see if there was a good way of getting all products. Finding the right path is a bit hard because the website only lets you navigate by category – so let’s try each category. Is there a way of getting all categories?
Well. Yes. As it turns out, if you ask for a category page that doesn’t exist, the response helpfully gives you a list of the ones that do exist:
That 404 in red is a special number that means “The thing you were looking for doesn’t exist”. But then in the “letter” (in the previous analogy) that was sent back, we’ve helpfully been told all the categories that do exist – here you can see “Featured” and “Fruit & Vegetables”. It’s also telling us how to access them (the “url” field).
Picking out a particular category, it turns out that you have to chase down multiple levels. I’ve ended up back at bananas:
A result! With a list of bananas. The results list is actually quite useful – it gives you all the information you expect to see, including cost and cost per unit – as well as the unit (kilos in this case). But that’s not enough – we need nutritional information!
Success! Unfortunately – there’s no nutritional information for bananas. A slight snag. Turns out bananas weren’t a good example. What is? What else? Pie!
Pie comes through
I’ve cheated slightly to go and get an individual pie from the new world website. It’s this Great Southern Pie. And here it is, a pie in all its glory – nutritional information and all:
Cleaning up
This is all quite messy. There must be a better way, right? Well yes. Going back to the page inspection, there are calls to api-prod.newworld.co.nz. That sounds much more promising!
This API needs us to identify ourselves. Fortunately, our web browser has to do this anyway and the calls to it are nice a clear. First, get simply get current user, which sends us back an access token:
Then we include that token in our new requests to the api:
Much faster, much cleaner. One small detail is that we need to tell the api (at least this one) which store we’re looking at. The Cromwell store is “9829c627-42c8-4ad7-b550-f9dba9fcd44a”. A list of stores comes back from the “api-prod.newworld.co.nz/v1/edge/store/” address.
Summary
Now I have the beginnings of a way to access product information programmatically, I can start to make a computer do all this work for me. From there – it’s a case of trying it out for new grocery stores, too.
Footnotes
Ackshually it’s not that simple. But for now – yes it is. ↩︎
I was wandering through a Sunday market here in Cromwell – you probably know the type. Incredibly expensive cheeses. A coffee cart that’s inexplicably about 50% more than the coffee shop a stone’s throw away. Someone selling hand-knitted tea-cosies and polished gemstones. The odd stall proudly owning a “witchy” label selling incenses, stones, and herbal teas. Maybe some soap. Not the kind of place you’d expect to get cheap food.
Tucked away in the corner, however, someone was selling sack of walnuts – in shells – for $12/kg. I didn’t know quite what the ratio of edible walnut-to-walnut shell by weight a walnut is, nor did I consider looking it up (a lot of times in my life you could ask “what were you thinking?” and the response would be: “Uh. Nothing?”).
Lugging a kilo of unshelled walnuts home, I promptly put them in a cupboard and forgot about them until my existing supply of (shelled) walnuts ran out, which took about 3 weeks (longer than anticipated).
Van life on the cheap necessitates two things:
Not buying much stuff
Not having much stuff
As such, I don’t actually own a nutcracker. The best I had, or so I thought, was a rubber mallet that I use to hammer in pegs for the awning. Not particularly hygienic but so long as it only touches the shell – so what?
Well. Hygiene was the least of my worries; I spent a good five minutes picking bits of walnut, shell or otherwise, out of the rug. And so this time I changed tack…
A hand job
“Can you crack walnuts by hand?”
Yes. Yes you can. It requires a bit of technique – namely lining up the seam of the walnut shell with your palm, going sideways, then pressing hard. Judging the pressure is a bit of a knack and many walnuts turned into more of a walnut powder than a walnut piece, shell included in the mix. But, nonetheless, it did end up being more successful than the mallet.
Briefly. About 5 nuts in I discovered what you might call “a tough nut to crack”, which resulted in deep imprints in my palms and a fully intact walnut. Cue switching to the base of a mug to crack them open, which turns out to be much more controllable and less painful – although I was concerned about the structural integrity of the mug.
Ultimately I spent about 30 minutes in all cracking 198 g of walnuts, including picking out what I can only describe as “walnut piece dividers” that sit through the middle of the nut itself. What did I produce for this effort? 93 g of edible walnut.
Some was wasted during early efforts but ultimately most walnuts were cracked with ease and the nut generally intact – I’d estimate that a skilled cracker could get around 50% yield.
The final cost
What does this mean? It means that so long as you don’t count your own time cracking them (and it’s a perfectly pleasant activity if you’re watching TV) the cost of walnuts, bought whole and shelled, becomes around $24/kg. The cost in shops? The cheapest I’ve found is $26.60/kg but the typical price is more like $30/kg – $40/kg.
Accounting for taste
A bonus here is that I think that these walnuts are significantly tastier than the price equivalent or cheaper in the super market. No doubt this vendor is seasonal and won’t be around for much longer, but I shall definitely be returning. The overall nutty flavour was much more prevalent, with much less bitterness, and they didn’t dry my mouth out as much
Energy budgets
Nuts may not be the obvious choice for cheap eating, but they’re packed full of energy – a good reason they’re typically included in trail mix – and are incredibly versatile. Mix water and sugar to make a simple syrup and candy them. Blitz them to make a powder and use as a flour-like substitute (other ingredients also required). Plus, packing around 65 g of fat and 15 g of protein per 100 g of walnut, the energy density is around 700 kcal/100 g – which makes my hand-shelled walnuts good for around $290 kcal/$ – that’s 67% higher than the pie in my previous post.
Humble pie. That which we are forced to eat when humiliatingly wrong. But also one of my favourite foodstuffs! The secret to a great pie lays in the secret ingredient of… pie. It’s hard to get a pie wrong (although if you serve it with mashed potatoes and/or peas on top – congratulations, you’ve managed).
It’s also the first thing I’d reach for when I want a quick, low fuss meal. New Zealand is great for pies (barring aforementioned serving style) and a good pie can be got for dirt cheap. Take this family pie from Fresh Choice – unfortunately Fresh Choice don’t display nutrition labels on their website but fortunately I happen to have one in the fridge. One $10 pie contains 1,735 kCal.
N.B. They recommend three servings, but depending on how you vary your meal sizes, I’d make it two for a large lunch, or three for a smaller dinner.
A price comparison
That’s 174 kCal/$ – how does that stack up? A famously student – and therefore cheap – meal of beans on toast should be a worthy contender. Let’s try to get 900 kCals of beans in our bellies!
I’m going to look at two options – the first is the absolute cheapest beans on toast possible. The second uses seeded bread and slightly more expensive beans, to give a slightly better meal (although you really shouldn’t be relying on this for particularly good nutritional value). The bread’s will be:
For beans, I’ll pair Pam’s to Pam’s, and Freya’s to Wattie’s (I’m desperate to compare to Heinz – the _correct_ brand of baked beans, but given this is a blog about eating cheap, I’ll have to draw the line):
I’m making an assumption that a listed serving of bread is two slices – it looks to be about right based on how many slices are in a bag. That means per tin of beans, with four slices of bread, we’re looking at:
Pam/Pam – 662 kCal per four slices or 1 tin of beans – for a cost of $1.37
Freya/Wattie – 874 kCal per four slices of 1 tin of beans – for a cost of $3.86
Adding an extra pair of slices of bread and half a tin of beans to the Pam/Pam combo to bring it up to 993 kCal gives a final price of $2.06.
I consider the range of 874 – 993 to be good enough for this, considering you’re not going to be eating half a slice of bread with a spoonful of beans.
Bean-to-pie
If you have the absolute cheapest beans, with the absolute cheapest bread, then you’re paying around $2 for nearly half of your daily calories (assuming you’re around 2,000 kCal). That’s incredible value, but unlikely to be sustainable as a long term diet without incurring some major issues.
If you want to add a bit of flavour* then you can move upmarket, get a bit more fibre and spend $3.86 for about the same. It’s a big step up, but with that you get some more protein, and around twice the fat. Not to mention the fibre – which really is important. Fibre isn’t listed on the online nutrition info but for the macros you’re looking at:
Combo
Protein (g)
Fat (g)
Carbs (g) / Sugars (g)
Pam/Pam
39
6.6
179.1 / 35.7
Freya/Wattie
43.6
13.2
95.9 / 35.4
Meanwhile our pie is coming in at $5 for a ~900 kCal portion. That yields:
Pie
Protein (g)
Fat (g)
Carbs (g) / Sugars (g)
BoB Creamy Chicken
30.6
62.4
37.2 / 2.7
I’ll admit, I was surprised at how low the protein was. But equally, pay a little extra and you’re getting around 1/8th the sugar. Not to mention the semblance of vitamins – this pie contains hints of carrot, onion, and celery – for a broader range of vitamins than will be found in just beans on toast. Again – I am not a nutritionist, but you’ll find more of vitamins A and K in foods with carrot and celery, without missing anything from the beans (thanks onion).
Whichever way you slice it
Ultimately the choice is yours. At these levels, for a quick and easy meal, I’d opt for the $5 pie portion over $2 of cheap beans on toast, every time. That’s the difference between around 15 minutes of work (at minimum wage) and around 6 minutes.
I was doing some shopping today and stopped to check my choice of sour cream, which I use to add an energy boost to my salads. Astounded at leaving with 6 items having spent nearly $90 (although I do have 3 weeks worth of chicken lunches in the freezer now) I went back over my bill to see where I could save.
In particular, I noticed that of the 6 things I bought, sour creams had a huge variation in cost-per-calorie. If you’re calorie counting, it really pays to pay attention to what’s actually in what you’re buying.
In this example I’ll compare three different brands of sour cream available at new world, in 250g sizes, as well as two “lite” alternatives. I’ll include costs for pre-and post-clubcard discount.
The contestants
First up, is Anchor Original Sour Cream, sporting the following macros (per 100g):
3.8 g protein
22.3 g fat
3.8 g carbs (of which, 3.1 g sugars)
953 kJ / 228 kcal
Anchor Original is my go to choice. A 250 g tub costs $4.45, or $3.99 with clubcard – for a cost of $1.78 and $.160 per 100 g respectively. Time to check and see how it holds up…
Our other contestants are:
Meadow Fresh Traditional
The Collective Straight Up (squeeze bag instead of a tub – likely a premium for that)
Anchor Lite
Meadow Fresh Lite
Sour cream – in numbers
Here’s how they stack up:
Brand
Protein (g/100 g)
Fat (g/100 g)
Carbs (g/100 g)
Energy (kcal/100 g)
Anchor Original
3.8
22.3
3.8
228
Meadow Fresh Traditional
3.5
17.6
5.9
194
The Collective
4.2
12.3
6.8
154
Anchor Lite
4.0
11.4
4.6
136
Meadow Fresh Lite
4.4
10.9
6.7
141
That’s a pretty wide spread – unsurprisingly, the “lite” variants are just less overall, but I was very surprised to see The Collective coming in with just 68% of the calories of Anchor Original. It does have one of the highest protein counts – and protein can lead to feeling fuller, so if you’re struggling to stay within your calorie target, that might end up being the better choice.
But how much do those creams cost, and what’s the cost per calorie?
Pricing it up
In the table below, I’ve calculated the cost per calorie for each option. Where a clubcard discount is an option, the reduced cost is in square brackets []. Because the costs are very low per kcal, I’ve priced it per 1,000 kcal for readability:
Brand
Cost ($/100 g)
Calorie-cost ($/ 1000 kcal)
Anchor Original
1.78 [1.60]
7.8 [7.0]
Meadow Fresh Traditional
1.88
9.7
The Collective
1.98
12.9
Anchor Lite
1.78 [1.60]
13.1 [11.8]
Meadow Fresh Lite
1.88
13.3
That’s quite a swing. From the most expensive cost-per-calorie (unsurprisingly a “lite” variant – Meadow Fresh Lite) to the cheapest is an increase of 70%. The figures here may seem negligible, but if this applies across even half your shop, by cost, that’s a significant expense. Again, if you’re calorie counting.
If you’re always using a fixed amount of a given ingredient then the cheapest per 100 g (assuming you’re on a tight budget) will be the easy choice.
In actual fact I’ve managed to not prove the thing I was trying to set out to show – that sometimes the more expensive option is cheaper in the long run. But hopefully you’ll agree based on this, that it’s entirely possible.
Even though I knew that there was going to be a big swing in cost-per-calorie, I was surprised to find it was as much as this.
In practical terms
It’s not practical to inspect the nutritional value of absolutely every item you buy, nor is it reasonable to expect someone to do so. It would take so much time to do a weekly shop that the prices would probably have changed!
But we can change that. Talking technically for a second, it’s entirely possible to automate this. I don’t mean some pipe-dream, I mean that given a free weekend, fetching prices and nutritional information for absolutely everything in your weekly shop, and making it searchable, is entirely feasible. This is something I’d like to consider making, freely available, in the future.
Wrapping it up
I hope this was informative – and shows just how much different products that may be nominally the same can differ, and that with the right circumstances, big savings could be made with careful choices.
I’m Mike Young and this is my attempt to feed myself for as little as possible, and stay healthy while I do it. I hope this may be of use to you, too!
I’m a mechanical engineer-cum-software engineer on hiatus and traveling. I’m privileged enough to say I’ve never had a minimum wage job, and never really needed to deal with sorting through groceries to keep costs low.
Until now…
I’m still in a pretty good position. The job may be minimum wage but other costs (namely rent and utilities) are very low. I’m also starting this from a baseline of having no dietary restrictions, children, other dependents, or health problems. Consequently, that leaves me with time enough to dive into what I eat.
Health and nutrition
I am not a medical professional. I don’t intend to provide medical or nutritional advice. But I am good with numbers, and in a pinch I can put together tools to help sort the wheat from the chaff.
I aim to make it clear what’s my opinion, what’s my opinion with a basis behind it, and what’s a direct quote from literature. For the most part though – this is just my opinion. I’ll also make it very clear where AI has been used to search sources – and I will always link the underlying sources when possible.
I think it’s important to watch what you eat – but more importantly to read the labels. Lots of cheap food is low in calorie density and comprises mostly of water. Investigating labels and comparing prices is time consuming – and I assume most people on minimum wage don’t have that time. I aim to do some of that work for you, and possible help along the way.
Macros
I am not a nutritionist. I cannot advise anything other than speaking to a health professional about what you should be eating. With that said, I generally follow a high-fat diet – it personally makes me feel satiated for more of the day. It’s worth noting that at the same time I started upping fat and dropping carbs, it coincided with a generally healthier diet anyway. As ever – consult a professional if in doubt.
Proteins – Help build and maintain muscle, and can provide energy when other sources are not available
Fats – Highly energy dense, slow to release
Each one has its place, and I consider it more important to look at the quality of the food rather than the pure content – 250 kcal of brown rice is likely to provide higher quality nutrition than 250 kcal of margarine.
Calorie counting
Is a major part of eating cheap. Over-eating means over-spending. There are a number of ways to know how many calories you should be eating – and it will serve you well to get a good baseline. Once you start counting you may be surprised at how calorie dense some foods are.
I had the good fortune to have a DXA [“Dexa”] scan 3 months ago. It’s absolutely not a requirement – but it was fun and quite interesting. From that, my estimated Basal Metabolic Rate (BMR – the amount of energy I need to exist) was calculated as 1,479 kcal. I add to that a rough baseline of 550 kcal for light activity and general walking, for a baseline of 2,140 kcal. If I do any strenuous work throughout the day – I’ll add more. For reference, I’m 175 cm tall and weigh 68 kg, a figure staying fairly constant.
Calorie counting can be done for free using a spreadsheet (or a piece of paper). It is much easier to use one of the variety of tracking apps – I can’t recommend any one in particular except that I use Cronometer which suits me well.
Beware – generic inputs on apps for foods may be alarmingly different to the label. If in doubt – trust the label where available.
Where from here?
I hope that gives a brief introduction of where I’m coming from. Over the course of this blog I’ll aim to look at the most cost and time-effective recipes for delivering nutrition. At the moment I’m in New Zealand, where the cost of groceries is high and the minimum wage is low – hopefully this is applicable to other countries.
At some point in the future I’d like to set up an app that scrapes supermarket websites for groceries and sorts out the cost-per-calorie, as well cost-per-<macro>calorie to help people sort brands and products to see what’s actually going to be cheaper, rather than cheapest per kilo.
Eating cheap 