When do you think there will be an Android app that could accurately perform real time object recognition?

In Star Wars Rebels, there was an E-XD-series infiltrator droid that could quickly take inventory of everything in a Rebel warehouse. With the advanced object recognition capabilities of modern AI, it’s only a matter of time before an app for Android can accurately and rapidly identify and store objects in real-time from video capture. This could be similar to a home inventory app where users only need to capture video and move around the house instead of taking pictures and labeling items. When do you think such an app will become available? Also, what is the closest app available right now?

edit: I didn’t say offline or on-device, I don’t know why everyone assumes that. I mean a service offered through an Android app.

DrownedRats,
@DrownedRats@lemmy.world avatar

Not likely in the near future, probably not feasible in the long term either. It’s not just about recognising an object. You could write a program that recognises a screw but you’d need far more complicated sensors and algorithms to identify the dimensions, specific characteristics, material composition, design specifications, etc, then apply that to every screw, bolt, washer, small component and assembly, tubes, threaded rods, tyres, pistons, brake pads, resistors, capacitors, diodes, seals, consumables, etc.

For a long time, I think that kind of thing would be wildly inaccurate, hugely expensive, massively complicated, and much less efficient than asking a human to kindly go over there and check all those things manually.

altima_neo,
@altima_neo@lemmy.zip avatar

Google Goggles used to be able to do that

Kolanaki,
@Kolanaki@yiffit.net avatar

Google Lens still can.

Zarxrax,

There is an app called Object Detector which does this. It’s not particularly accurate and can’t recognize a lot of objects though. It does run on phones in realtime though.

4am, (edited )

Frigate does this on a raspberry pi or Intel NUC already. Would be power hungry in a phone but if you are not training ML models and just looking for objects they already know, the tech would be ready today.

EDIT: Here’s Google’s article on how to create your own TensorFlow app for Android developers.google.com/ml-kit/vision/…/android

QubaXR,
@QubaXR@lemmy.world avatar

GPT 4 with image uploads gets pretty damn close, though it’s not real-time and processed server side

adespoton,

Think about what Apple currently has: dedicated ML processing chip with multiple cores, and yet the on-device object recognition is still an “overnight while plugged in” process for a single image, and only detects a limited number of object types.

Real-time mobile offline OR is still the mythical “at least ten years out.” It needs improvements in processors, sample sets, training data and algorithms to get to real-time.

ryathal,

An isolated phone, not for a while. A phone with a dedicated 5g connection would be pretty close.

breadsmasher,
@breadsmasher@lemmy.world avatar

Doesn’t google lens basically do this already?

over_clox,

Yes, but OP is referring to realtime object recognition. Although we don’t have to wait very long for object recognition right now, we still have to wait a bit. That’s not quite realtime.

dope,

Object recognition.

Then “action recognition”. I mean, what the objects are doing. To each other and such.

Then you have a narration machine. Which could be nice.

over_clox, (edited )

Very related, not my work though…

Self Aware Lara Croft

This seems like something of an ideal effort, but it also took lots of human work to even prepare the AI model, and many many runs to refine it.

We’re not even close to realtime object recognition at this point. Delayed recognition, yes. Realtime, no.

Edit: The self Aware Lara Croft series recently dropped their latest video of Level 7…

piped.video/watch?v=SYX4CwyZ1LM

over_clox, (edited )

Modern AI, as you’re seeing it today, is processed by massive data centers online with thousands of processing units running in parallel, not by your local device. Your device would be way too slow to expect any sort of realtime object recognition, at least with the current state of technology.

TL;DR - I don’t think it’ll happen anytime soon, at least not on your local device. It would take a super fast and steady connection to the AI service.

deegeese, (edited )

Don’t underestimate the potential for optimization when you can constrain the problem to a narrow range of uses. Model pruning and custom silicon go far. Voice assistants used to be purely cloud compute, but a lot of common use cases are done on device now.

over_clox,

Yes, I’ve been testing FUTO Voice Recognition lately. It’s awesome as hell, but it is far from realtime. And this ain’t even object recognition, it’s only voice recognition.

voiceinput.futo.org

play.google.com/store/apps/details?id=org.futo.vo…

qwertyqwertyqwerty,

Honestly, I expect some form of it in the next five years. Tech can move fast when it wants to and there’s 💵 involved.

over_clox,

Is Moore’s Law Finally Dead?

umbrella,

dunno, some mobile devices are starting to ship with pretty passable gpus nowadays

over_clox,

We’re not talking about image rendering, we’re talking about image recognition. Although they may seem related, they are not.

It’s one thing to sling a 3D model and textures to a GPU, but it’s totally a different thing to take a photo and sling it against a humongous AI model being run at a datacenter with billions of images to compare it to.

umbrella,

image recognition is also done on gpus, a powerful enough gpu on say, a phone can do a variety of ai tasks

a mobile integrated intel gpu can already do facial recognition on a video stream for example

data centers have to be big because they centralize a lot of work

over_clox, (edited )

Recognizing a face is one thing, that’s more or less just knowing certain geometries. Recognizing who that face actually is, or what model car that is, or whatever, requires processing through a huge database of information.

Also, as of right now, not all AI systems are even smart enough to distinguish a human from a monkey. They both have faces yo…

umbrella,

tell that to my frigate nvr

over_clox,

No shit Watson, that’s my whole point. AI as anyone today knows it is cloud based, meaning you’re tethered to the internet. Your device can’t process it all by its little measly lonesome self.

umbrella, (edited )

you should look up what frigate is.

my desktop gpu can generate ai art pretty quickly too

over_clox,

Again, we are not discussing generating AI, we are discussing recognizing images from a camera. That requires parallel processing and many terabytes, if not even petabytes of images to compare to.

You got a petabyte of storage and a 1024 core processor to scavenge through all those images to tell you that the picture of your butt plug looks like a purple booty packer 3000?

Akrenion,

Modern ai only needs those images to learn features and to project that on embeddings. I do not disagree with your assessment but you are talking about reverse image searching and not about neural nets.

over_clox,

You act like I don’t have experience programming neural networks. I actually do, starting from the early 2000s, initially for OCR (Optical Character Recognition).

If you want to do literal realtime object recognition, you’ll have to reverse search images, otherwise you’ll be dealing with a system that can only generalize the object, but can’t even tell if a face is a human or a monkey.

philstar.com/…/nbi-test-shows-sim-registration-sy…

umbrella, (edited )

again, look up frigate. its similar although not the same as what you describe.

you dont need that much hardware to do ai, im not really saying the average joe will train huge intricate models for personal use on his laptop/phone.

fartsparkles,

Don’t know about an Android app but YOLOv8 Detect and similar models can detect objects in videos and classify them.

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • random
  • uselessserver093
  • Food
  • aaaaaaacccccccce
  • [email protected]
  • test
  • CafeMeta
  • testmag
  • MUD
  • RhythmGameZone
  • RSS
  • dabs
  • Socialism
  • KbinCafe
  • TheResearchGuardian
  • oklahoma
  • feritale
  • SuperSentai
  • KamenRider
  • All magazines
    •