Frozen LLM-Generated Preprocessing for Medical-Claims Adherence Classification: An Internal Model-Sensitivity Study

Large language models (LLMs) can generate data-preprocessing code from data profiles, but their value relative to predefined expert workflows in clinical tabular classification remains unclear. This study evaluated whether frozen LLM-generated preprocessing matches a Manual Expert workflow for medication-adherence classification and identified the components responsible for performance differences. We analyzed 24,071 de-duplicated medical-claims records from 2022, comparing Manual Expert preprocessing against frozen code generated by DeepSeek, Claude Sonnet 4.6, GPT-5.4, and iterative CAAFE. Stateful transformations were fitted exclusively within training folds. Performance was assessed using repeated 5 × 10-fold cross-validation, a locked 20% holdout with 2,000 bootstrap samples, and Nadeau–Bengio-corrected equivalence testing. With LightGBM, all pipelines were equivalent in accuracy, Matthews correlation coefficient (MCC), and area under the receiver operating characteristic curve (AUC). Cross-validation accuracy ranged from 0.8204 to 0.8229, and every corrected 90% confidence interval remained within the prespecified ±0.01 equivalence margin; the widest interval extended from −0.0041 to 0.0009. Component ablation attributed the entire improvement over raw data to ratio features (+0.0018). Log1p transformations produced no improvement (+0.0000), whereas scaling slightly reduced accuracy (−0.0007), consistent with tree-based models’ insensitivity to monotonic transformations. Excluding the outcome-proximal variable UNITSTOTAL reduced holdout accuracy from 0.820 to 0.758 and AUC from 0.903 to 0.839, although pipeline equivalence persisted (accuracy span: 0.0018). However, equivalence depended on the estimator: naive Logistic Regression exhibited an accuracy span of 0.202, decreasing to 0.161 after uniform scaling and nested regularization tuning, with differences concentrated in CAAFE. Thus, frozen LLM-generated preprocessing can match expert-designed preprocessing for internal, tree-based medication-adherence classification. Gains primarily reflect feature construction rather than distributional adjustment. These findings also caution against assuming universal preprocessing superiority. Preprocessing strategies should therefore be evaluated jointly with downstream estimators, while claims-based prediction results require careful assessment of outcome-proximal information before being interpreted as genuine predictive performance.