Expensive phones like the most recent high-end Pixel/Samsung/Motorola/Xiaomi/Realme devices have embedded LLMs accessible to apps via Android’s Google AI Edge SDK / AICore, they can reply to prompts without using the Internet.
I am developing an app that uses this SDK to perform LLM-based tasks locally, for instance summarization and language detection. I wrote prompts for each of these tasks.
I would like to test my app’s prompts on all/most AICore-enabled devices. Testing is very important as each model presumably has a different level of performance, a different version of Gemini Nano, or even a different LLM altogether.
Unfortunately I don’t have all of these devices, they cost a lot of money and emulators can’t be used to test each maker’s embedded LLM.
I would like a crowdsource platform where people who own these devices can effortlessly run my prompts and send me the responses.
- Ideally it would be easy for me to manage the prompts, and to see the responses categorized by model/OS version/AICore version.
- Ideally the device owners would not need to do anything, just install it. Maybe run it once in a while if auto-running presents challenges.
- Ideally the crowdsourcing platform could be used by any other developer, however I am open to a one-off solution as well.
- Maybe it could be integrated into the app itself. At runtime it would detect whether this user’s LLM responses are valuable, and only then run it, ideally after asking the user for permission.
Ideally the platform would be open source, free to use, and the volunteers would be motivated by helping improve my app which is also open source.
But for-profit solutions where app developers pay money and some of that money is redistributed to device owners, are also acceptable answers. Some loyal users may also be willing to do it for free, maybe in exchange for in-app points or early access.
