Unveiling hidden permissions : an LLM framework for detecting privacy and security concerns in AI mobile apps reviews

Mobile AI applications enhance functionality but introduce complex privacy and security challenges. This research develops and evaluates an automated framework that leverages Large Language Models (LLMs) to analyze user reviews and unveil “hidden permissions” defined not as technically undeclared functionalities, but as declared permissions whose purpose or necessity is opaque to users, leading to perceived privacy risks. The framework integrates static analysis of permission manifests with a hybrid Natural Language Processing (NLP) pipeline that combines Term Frequency-Inverse Document Frequency (TF-IDF) with BERT embeddings. A fine-tuned RoBERTa model then classifies user-reported concerns into predefined risk categories. We correlate these user-reported behaviors with declared permissions to identify potential mismatches and prioritize them using a risk-scoring methodology validated against the MITRE Common Weakness Enumeration (CWE) database. In an evaluation against other LLM architectures (GPT-3.5, DistilBERT, XLNet, and LLaMA-2), our fine-tuned RoBERTa model demonstrates superior performance, achieving an F1-score of 0.90 in classifying reviews related to unauthorized tracking. The framework effectively surfaces and prioritizes user-perceived privacy risks, offering actionable insights for developers to address mismatches between an app’s declared permissions and its user-experienced behavior, thereby fostering a more secure and trustworthy AI mobile ecosystem.