Artificial intelligence and facial recognition in sheep

It's great to be here. Thanks, Josh. It's awesome to see you up here at Mumblebone, and it's a pleasure to be here with this great audience. I run a company called neXtgen Agri. I'm based in New Zealand, but I have a team of staff here. Henry Hickson, who you'll hear from next, is the local in New South Wales, but we've got a guy in South Australia, and we've just got Amy on in WA along with Georgia. So we're really starting to expand across Australia.

One of the things we've been doing is working on artificial intelligence. For about 500 million years there's been some sort of device on animals that could see things. That's the evolution of the eye. So for 500 million years the eye has pretty much had the dominant spot on seeing things. That's all changed in the last couple of decades, where cameras and computers combined can see the same stuff that eyes can. You're thinking, that's interesting, but I don't know why you're talking about it. The reality is that it opens up heaps of opportunities for ag.

I forgot to answer your question, Josh. The question Josh asked me to answer before I started, which I've already started, was what's the one thing we want people to take home from a talk like today. I think it follows on from Howard, and it's possibilities. As Howard said, change or adaptation only happens if you believe it's possible, and that's really where this project came from. A farmer talked to me about how we could have cameras that can see sheep and identify sheep, and I thought, I don't know, maybe we can.

So four years later, and I don't know how many hundreds of thousands of dollars, we can do it, and that opens up the opportunities.

What artificial intelligence can do

Before I go further into that, I just wanted to cover off what artificial intelligence, or machine learning, is. You'd have to be hiding under a rock not to hear that it's happening, but you'd also be wondering what it all means. Basically it's training computers to do things that humans have always done, and to do them much more efficiently than we could possibly ever achieve, because of the volumes of data.

What Howard was talking about before was that when we go into a shop and we're confronted with a heap of brands, we can't possibly know everything about every production system and all the things we'd need to know to make an informed decision. But artificial intelligence can just swallow all that information and spit out the result.

It's got amazing potential in agriculture, and obviously across everything we do these days. It doesn't matter where you're hiding, AI will be controlling something you do. Whether you pick up your phone or jump on your computer, wherever you are, artificial intelligence is controlling part of your decision making, and that can be pretty scary. But it's also really enabling. It means we don't have to know everything.

Anyone in the room who's a uni student will know what ChatGPT is. These are a few examples of the kind of things out there. I get a newsletter every day that gives me the top 10 new companies doing something in AI, and this stuff is galloping out of control. ChatGPT is pretty much a model that's been trained by swallowing the entire internet. I can write in there, "Give me a five-page essay on mulesing in sheep," and it will search all that stuff and give me a really rational response back in about five or 10 seconds, written like I would write it, if you train it well. So it's an amazing power.

If you put into Midjourney, "Give me the royals doing yoga," you get Prince Charles, or King Charles these days, in a yoga suit. If you blew that up, it's lifelike. It's a very high-quality image of something that bloke's never done, and that's the power of AI.

When we originally thought artificial intelligence was on its way, we thought the arts would be the best place to be, because that would be the thing humans could still do. But artificial intelligence is way more creative, way more powerful and way faster than any artist on the planet. So when Howard talks to you about truth, you have no idea what's true anymore. If you see an image, you have no idea if it's been created by Midjourney or by someone with a lens. That creates a really interesting time for us as humans.

I'm going to get to sheep in a minute, but the reason I'm going through this is that if we don't believe this stuff is possible, we never challenge ourselves enough to innovate. It was only when I knew this stuff was possible that I started to innovate, and it's the same whether we're talking about breeding or our pastures. Change and adaptation only ever happen when you believe they can, and when you believe there's something out there that can help you. If we don't know the power of this stuff, we'll never actually use it.

I work every day with farmers and the farming industry, and it's the most innovative bunch you can come across. So if we can put the knowledge that these tools exist in your heads, it means you will come up with fantastic solutions, and that's what excites me about this whole technology.

Can cameras tell sheep apart?

It will revolutionise agriculture. It already is, but not to the extent that it will. That's just a simple model that counts sheep, which is probably low-hanging fruit. That's not ours running there, but we've got a model that counts sheep as well, very accurately, and identifies those animals.

Every day in saleyards across the nation, in Victoria for example, there's around one and a half per cent error between the truck count and the EID count. You've got a couple of humans counting, and you've got EID counts, and no one knows if it was a misread EID, or the truck driver or the stock agent made a mistake, or where that error is. It's a massive wastage in our industry. That's a really simple example, but there are things we can do that will change the game for us.

We're in an audience full of sheep producers, so this is obvious to you, but if I'm in an audience of other humans, or cattle producers, it surprises the hell out of them that sheep don't all look the same. Everyone thinks they're exactly identical, but the reality is that all of those animals have subtle differences, the same as we do, and those differences can be detected by computers.

When I entered the country 10 days ago, I walked out of New Zealand through a gate that uses artificial intelligence to work out who I am, and I came into a gate in Sydney that did the same thing. The models checked my facial features, and bang, it lets me into the country, knowing that I'm matched to that passport. The same thing can happen in sheep, and we'll get to why that's important, because they're not going to go through customs very often.

So our first job, four years ago, was working out whether this is possible. Can the cameras tell the difference between sheep? The answer to that question is yes. That was work we did with AWI about four years ago. We then had to redo that work over the last four years to use better models.

The interesting thing in this game is that the technology is moving so quickly that the cameras we've got on farms today in Victoria, and the computers driving those cameras, didn't exist 12 months ago. The exciting thing about that is that the ag industry doesn't have to invest in that technology. It's not coming from us putting our hard-earned levy funds into that development. The driver is the demand for GPUs, or graphics processing units, which drive all this technology, and that's kids and adults sitting in cities playing computer games. That's why it's moving so fast, because people like playing things that look like real life on their computers. That's not something I'm that interested in, but I'm pretty excited about the amount of effort and investment going into that area, so we can use the fruits of that work.

So we've got a model that can identify individual sheep at about 98 per cent accuracy, and it'll get better than that. We've done that over the last little while. I've got a team of four engineers. The other cool thing about developing technology in ag is that you get people who have never been in ag before. Four years ago these engineers wouldn't have known which end the tucker goes in on a sheep, yet now they spend every day looking at sheep on a computer, which sounds a bit tough. But they're absolutely driven and passionate about the solution they're providing for the industry.

It's so awesome for them, because the other option for them is to do cyber security for a bank or some finance company, doing some AI to tweak the last little one percenter out of that. Whereas they're working on something real, which goes to Howard's point. They absolutely love being involved in ag and the opportunity to work on something real. There are lots of people in technology who are lost in their nine to five jobs, who have an amazing amount of knowledge and power but are doing jobs that really don't stimulate them.

Cameras in the paddock

In terms of what we're doing, if we think about the sheep industry, and these are numbers I've made up roughly, one person managing five thousand sheep wouldn't be outlandish. The opportunity for that person to see what's going on in every paddock is impossible. We've got so much work to do. But cameras can be there, and cameras can see what they can see.

So where this technology starts is, do you see what I see? If we stood in our paddocks every day, we'd know exactly what's going on there. We just don't have time to do that, and we're never going to, and that's where this technology can take over.

Essentially, you can train cameras to see things. You can train them not just to see which animal it is, but also to see if it's limping, or showing flystrike activity, or rubbing against the fence. All those things that, if we were driving through that paddock, we would stop and see and go, "Oh, we might have a problem there." We've seen a little bit of lameness in one sheep, so what does that mean? Or we've seen a sheep biting its backside, so there's a bit of fly around.

We're going into this uncomfortable world where it gets harder and harder to get labour on farms, and highly trained staff who would see that stuff. You can train anyone to run a feed cart into a paddock, but to actually see the subtle things that you all as stock people know, it's getting harder and harder to have that level of knowledge in the paddock. That's where this sort of technology can kick in. It's not there yet, but my job today is to tell you what's possible, so you can start thinking about it, and ring me up and give me some new ideas.

So things like how much feed is there, the health and disease of those animals, what the lambs are up to. Not detecting pregnancy status, but also just whether that sheep should be in that paddock at all. How many people have had out-of-season lambs in their lives? There are a lot of liars in this room, Howard, a whole heap of liars in here.

There are things this technology will be able to do. When we looked at this opportunity for us in the industry, there's a range of ways we can get value out of cameras on farms. Targeted management is up there, and that's real-time management. Better genetics, so identifying our most productive ewes, which is where we're going first. Less labour and better pasture utilisation are the big wins for how this technology will change our industry.

Matching lambs to their mothers

Take the untimeliness of shifting paddocks. It seems like a really simple thing. You put the sheep there, then you open the gate and shift them. But we get that wrong a lot across the industry, and the few days we get it wrong, because we don't shift them early enough or we shift them too early, add up to massive potential for the industry.

Where the technology has started is working out which lambs belong to which ewes. Across stud flocks, that's a major limitation. Everyone's using either DNA, or mothering up in the paddock, or mothering up in the yards, and that's a lot of labour and cost. The aim of this technology is to do that job for them: work out who the lamb is, who mum is, who's hanging around each other, and therefore how they belong.

If we look in that paddock, we can see the pairs. We can see who's hanging around each other, but the reality is we can't see what that ewe's number is. You can put tags in their ears or whatever, but there's a lot of work and a lot of labour in that. So the job of this technology is to have a camera that does that work for you. It's like having a shepherd, which is a New Zealand term, or a stockie, in that paddock as long as the sun is up.

But our main focus is to enable really supercharged EID. When people hear about facial recognition in sheep, they think it's going to get rid of EID. That is not our aim. Our aim is not to replace EID. Our aim is to supercharge EID. Anyone who's got EID in their sheep has got some information, but there's one key thing you don't have, and that's which lamb came from which ewe.

That single limitation reduces the value of EID by a significant proportion. In a flock of sheep, we know who scanned twins into that mob, so we put all those twin ewes in a mob and we know they all went in there with twin lambs on. Then we drafted those lambs off at weaning, and we've got some little monkey of a lamb we don't like that much, a couple of absolutely fantastic lambs, and a range across that. And we have no idea which lamb came from which ewe, or which were our most productive ewes.

We've got this histogram of ewes in there. One has produced a couple of 30 kilo lambs, and these are inflated prices compared to where the market's at now, but we'll get back there. Then you've got another ewe that's produced 80 bucks for the year. Right now we have no idea which one's which, and in a commercial sense we will never know. Our studs do, because they spend the money to find the answer, but commercially we have no idea, and without that bit of information the value of EID is significantly reduced. So that's where we started.

How the camera system works

To bore you with a bit more tech, we're running five different machine learning, or artificial intelligence, models on a camera. It's running on a computer about that big, which is a pretty cool bit of kit that, again, didn't exist 12 months ago.

The computer is sitting in a paddock on a pan-tilt-zoom camera, so it's looking around. The first model says there's something there. The second model says it's a sheep or a lamb. This is all completely automated, with no humans involved. It then detects the face, crops that face out and orientates it so we know which way it's looking. Humans have a flatter face than a sheep, so the angle you're looking at a sheep from matters a bit more. The next model works out who it is, and the next model works out which other sheep are around it.

It's doing that completely automatically. We've got four blokes in New Zealand who sit there and watch the camera do its thing, and they're retraining the models all the time and loading them back onto the device, but it's running autonomously. That sounds pretty easy and pretty quick to say, but it's been a hell of a lot of work to get that far, and that's where we're at today.

To give you an example of what that looks like, this particular camera is over in New Zealand. This is the model running through. The blue boxes are lambs and the black boxes are ewes. It's working out who they are, zooming in and zooming out. It's got a 60 times optical zoom on it, so it can see about 120 to 150 metres now.

One of the things about doing this work is that you've got to track these animals, because it takes a while for them to look at the camera. They don't just smile and turn to the camera. You've got to wait for them to do that, so you've got to keep tracking them, because you want to know that these two are together. We'll know who that lamb is, but we have to wait for mum to look up to see who that ewe is. It's an insane amount of power that's captured in these computers.

That computer will keep tracking. This is an old model now, but it'll keep tracking that lamb as it walks off and make sure it holds it until we get the information we need. Then it moves on to the next one. That's sitting in the paddock at my brother's place at the moment, and we're continuing to improve it, with the aim of matching up lambs and dams for all commercial farms.

When we look at that, there are 73,000 sheep farms across Australia, 94 million sheep, 1.2 billion sheep across the globe and three billion ruminants across the world. So there's a big market there for us, and that's the reason we're tackling it. I think it will definitely enable massive change in agriculture. Whether we're the ones who crack it or whether we go broke in the process, I don't know, but we'll know in time.

In addition to the productivity benefits, we're already seeing interest from premium markets. Like Howard says, people want to be guilt-free, and brands have to protect themselves from guilt. With technology like this, they can say, "We've got cameras in those paddocks watching those animals 24/7, and we've got farmers who act accordingly." That is pretty sexy to some brands, and we've already been in discussion with a group in the UK around that. I think this technology will help us as farmers safeguard ourselves against what people might think we're doing badly. It enables us to really certify that we're doing absolutely best practice and always monitoring our animals.

What else AI could do on farm

I'm pretty excited about this technology, in case you couldn't tell. I think it changes everything, and not just what we're doing. The ability for cameras and computers to be linked, to think and to see and to do things on farm is going to be insanely powerful. We are only limited by our imagination.

So knowing what's possible is the key thing. When we talk about change, and the lack of change in our industry, it's not the inability to do stuff, and it's not because we're not educated. It's because we don't know what's possible, and we need to really just imagine.

There are things like, can we automate preg scanning? Anything where a human and an image are looking at each other, can we not do that with AI? That's not about replacing preg scanners. It's about really improving their accuracy, particularly their ability to determine the age of the foetus. All of that's going to come to our industry over time.

We do a lot of muscle and fat scanning, and you can do about four or five hundred a day because of the time it takes. If we can put an AI model on that, we'll smash that out as quick as we can get a scan. And I'm sure we'll do live animal IMF scanning. There are just so many opportunities. There's already a company out of Adelaide with a small camera-based micron measurement built on AI technology, the same stuff I'm talking about. They've trained the model so you basically put it on the wool on the sheep and it gives you the micron immediately.

All of this stuff is coming, and what I would ask is that you embrace it, because it will change the game for us in a really positive way. There's going to be lots of bad stuff happen to us because of AI too, mainly in our social media. All that stuff you get fed from ads is AI algorithms feeding you stuff based on what you look at. So don't slow down on those half-naked girls, lads, because then you'll get lots of that stuff in your feed. You've got to be careful what you slow down on, because the AI models are watching, and they'll keep feeding you the stuff they think you like, whether you do or not. We're not going there.

The technology is in a separate company as of the middle of last year. It was great to be able to bring those engineers in with some equity, so that's going along. We've been fortunate to get some support from the New Zealand government and from a private firm. That's my talk on AI. I'm going to move now to footrot, which is way more exciting than that stuff.