I’ve been using it for a while now on my Pixel and I gotta say it’s actually really useful, you can copy text from anywhere, even apps that normally won’t let you (both intentionally blocked and unintentionally) copy/select text
And the easy “translate what’s on my screen” functionality has been really good on Lemmy when I come across the random German memes lmao
The actual search function also comes in handy when I want to reverse search an image from an app, but the app won’t let me save it. It’s nice to skip a step or 2.
That being said, since it’s actually good I now expect it to be nerfed and/or abandoned in about 6 months
You must log in or # to comment.
That being said, since it’s actually good I now expect it to be nerfed and/or abandoned in about 6 months
Exactly this, Google is where it is because generally its products start out pretty good
Do you know if there’s a FOSS project that does any of this? Android does provide a screen capture API, and there are some FOSS OCR and translation apps separately, bit I don’t know of one that combines these things.
Not sure what CtS is using, but I don’t think it’s the normal screen capture API, because it works in apps that block screenshots
isn’t it just a new way to access screen based search? i mean, i can do things you tak about by launching google assistant (non-gemini) and choosing “search on my screen”, i use it daily to translate difficult japanese in my dulingo, select text and search on my non-pixel/samsung phone, it’ there since few years AFAIK, i don’t think it’s going anywhere, because reverse image search is the future, also you’re training their neural networks that way(👉ﾟヮﾟ)👉
for the most part it’s just a finer, more intuitive interface to that exact function, imo. seems like mostly just marketting, for what’s been there the whole time
Search on my screen stopped working reliably for me a while ago, and it didn’t work in apps that are blocking screenshots/sharing
The ‘translate what’s on my screen’ was a thing that google assistant could do around Android 8. I found it very handy when living abroad and not really knowing the language.
That one didn’t use OCR, but could only do text in UI elements. Don’t know why they killed it. Maybe some security reasons caused the API that reads another apps UI elements to be canned.
It’s really helpful as someone trying to learn a language since I can just use it to select words I don’t know and immediately translate them or look them up in a dictionary.
I think it has gotten better since it was first released for Samsung. The new translate button is very good.
Even if its nerfed and abandoned, a heartening new trend that I’ve seen is cutting edge technology being democratized in FOSS apps. There’s all sorts of ai stuff and image stuff that was limited to the top five tech companies a year ago that’s now available to everyone.
What I like the most is the music recognition button, because its really quick and convenient, and it also works for content playing on your phone, as long as you use the phone’s speakers. I think it doesnt use the media volume but rather the microphones only, but having the phone’s volume enabled it probably feeds back into the microphones and it works this way.
