Open-Source Intelligence for Code Provenance and the Security Patterns that Separate Human and Large-Language-Model Implementations of Common Programming Tasks

Image generated by AI

TLDR

• What: An open-source intelligence pipeline that uses lightweight, language-agnostic style and security features to classify code provenance (human vs. machine) and map distinct security pattern divergences.
• Who's at risk: DevSecOps pipelines, automated code review assistants (e.g., commercial assistants like GitHub Copilot), software supply chain auditors, and organizations relying blindly on LLM-generated security repairs.
• Key number: A style-based random forest classifier distinguishes human-written code (from Stack Overflow) from LLM-generated code with 93% accuracy, while LLM vulnerability repairs suffer from a "partial-fix" failure mode in 16% of cases.

As software engineering teams rapidly shift from manually copying snippets on Stack Overflow to generating production-ready logic directly within IDE assistants or via model APIs, the security properties of the software supply chain are fundamentally mutating. While prior research has focused on overall code quality or single-model vulnerability rates, a deeper question remains: can we reliably identify machine-generated code from its stylistic fingerprints, and how does its security profile actually diverge from human-authored code? This research addresses these questions by proving that lightweight, interpretable stylistic features can classify human vs. LLM provenance with remarkable accuracy, while exposing a critical "completeness hazard" and "partial-fix" failures in automated LLM vulnerability remediation.

Threat Model

Background / Problem Setup

The developer's process of sourcing code has split into two paradigms: human-curated archives (e.g., Stack Overflow) and generative language models. Security research has historically examined these domains in isolation.

                  ┌──────────────────────┐
                  │ Sourced Code Snippet │
                  └──────────┬───────────┘
                             │
              ┌──────────────┴──────────────┐
              ▼                             ▼
   Style Fingerprinting           Security Pattern Mining
  (Provenance Attribution)        (Divergence Analysis)
  - Raw size of sample            - Library-backed defense (bcrypt)
  - Presence of functions         - Parameterized queries
  - Structural imports            - completeness hazard (inlined secrets)

The table below contrasts the positioning of this paper's methodology with prior works:

Methodology

1. Data Collection

The author built a fully reproducible, data-driven pipeline (shown in Listing 1) to collect code across 31 security-sensitive tasks (subjects) spanning 5 languages (Python, JavaScript, Go, Java, and HTML).

• Human Corpus ($SO$): 117 highly-voted code blocks retrieved via the public Stack Overflow API.
• Model Corpus ($LLM$): 411 code blocks generated by 9 open-weight language models (e.g., Codestral-22B, Mistral-Large-3, Nemotron-3-Ultra, Tencent-Hy3) accessed through a local gateway.

2. Feature Extraction

Each code snippet is reduced to two distinct families of features:

• Style Metrics: Language-agnostic structural properties including comment density, blank-line ratio, number of import statements, total code lines, average line length, and indicators for error handling or functional blocks.
• Security-Pattern Checks: Deterministic regular expression checks tailored per task (e.g., checking for PKCE, state parameters, parameterized SQL vs. string concatenation, and weak hashes like MD5/SHA1).

The core security metric for any given sample is defined by a single security score ($sec$):

This score strictly bounds values between $[-1, 1]$, where positive values indicate a strong adherence to defensive security practices.

3. Classification and Remediating Vulnerabilities

To demonstrate the practical utility of these features, the author evaluated:

1. Provenance Classification: Training a Random Forest classifier over the style metrics and the two aggregate security fractions.
2. Vulnerability Repair Case Study: Presenting the 9 models with 21 vulnerable code seeds across 12 distinct Common Weakness Enumerations (CWEs) to evaluate their direct remediation capability.

Code Collection & Feature Extraction Pipeline

# Stylized representation of the data pipeline defined in Listing 1
def pipeline_execution(specification, usable_models):
    samples = []
    for subject in specification:
        # 1. Fetch human answers
        human_code = stackoverflow_api_fetch(subject.query)
        samples.append((human_code, "human"))
        
        # 2. Query open-weight models
        for model in usable_models:
            model_code = query_gateway_chat(model, subject.prompt)
            samples.append((model_code, model))
            
    # 3. Deterministic feature extraction
    features = []
    for code, origin in samples:
        style_feats = calculate_style_metrics(code)
        security_feats = run_regex_detectors(code, subject.checks)
        sec_score = len(security_feats.secure) / total_sec - len(security_feats.insecure) / total_insec
        features.append((style_feats, sec_score, origin))
        
    # 4. Perform cross-validated classification
    attribution_results = cross_validate_classifier(features)
    return attribution_results

Key Results

1. Provenance Attribution Performance

Stylistic features dominate the ability to distinguish who wrote a code snippet. As Figure 1 shows, features like the number of lines, comment ratio, and the presence of functional blocks carry the highest permutation importance.

2. Security Divergence Results

The models exhibit a completely different security profile compared to the legacy code patterns dominating Stack Overflow.

As summarized in Table 4, model-generated code is significantly longer (an average of 50.1 lines vs. 14.3 lines for Stack Overflow) and more defensively wrapped (error handling is present in 31.4% of model samples vs. only 8.5% of human answers).

Security Score Distribution Comparison (as shown in Figure 5)
============================================================
Human (StackOverflow):  [+0.12] ▓▓░░░░░░░░
Models (Aggregate):     [+0.40] ▓▓▓▓▓▓░░░░
  - Nemotron-3-Ultra:   [+0.62] ▓▓▓▓▓▓▓▓░░  <-- Safest baseline
  - Codestral-22B:      [+0.26] ▓▓▓▓░░░░░░
  - Qwen3-Coder-30B:    [-0.40] ░░░░░░░░░░  <-- Worst performer

The research shows that models reliably lead in adopting modern standard libraries (e.g., automatically utilizing bcrypt for password hashing as detailed in Section 10.1, or implementing parameterized queries to completely eliminate string-concatenated SQL injection points).

However, LLMs suffer heavily from the "completeness hazard": because their objective is to produce runnable, copy-paste-ready code blocks, they often inline literal placeholder secrets (e.g., hardcoding a client secret key in OAuth or session management tasks, as shown in Section 10.4). Humans, writing shorter fragments, leave these configurations blank or direct the reader to externalize them.

3. Vulnerability Repair Performance

When handed vulnerable code blocks and asked to fix them, the models achieved a 77% overall repair rate.

The Partial-Fix Hazard

In 16% of all vulnerability repairs, the model successfully stripped out the insecure pattern but failed to add the necessary secure defense. For instance, in a password hashing task, a model might delete the weak MD5 hashing call but save the password in plain text. A naive signature scanner would flag this code as clean because the insecure pattern is gone, even though the application remains deeply vulnerable.

Limitations & Open Questions

• Lexical/Regex Deficits: The paper's framework utilizes regular expressions for security-pattern detection. While highly reproducible and transparent, lexical patterns do not guarantee semantic security. For example, a sample can adopt bcrypt (triggering a "secure" regex match) but call it with an insecure work factor or implement it within a broken authentication flow.
• Exclusion of Closed-Source Frontiers: While open-weight models represent a major portion of developer workflows, industry-dominant closed-source commercial assistants (such as GitHub Copilot) were not directly evaluated under the local gateway.
• The Temporal Confounder: Stack Overflow highly-voted answers accumulate score over many years. This biases the human corpus toward outdated defensive recommendations (e.g., pre-dating modern CORS policies or password hashing standards), inflating the comparative security score gap.
• Prompt Fragility: Every LLM was queried using a single neutral prompt. Changing the system instruction to explicitly demand highly secure code would likely shift model safety scores upwards.

What Practitioners Should Do

1. Route Code Audits Based on Style Provenance

Deploy a lightweight style classifier in your CI/CD pipelines to flag the provenance of incoming commits.

• Code flagged as human-authored should be routed through legacy omission checks (e.g., scanning for missing input validation, outdated cryptographic libraries, and string-concatenated SQL).
• Code flagged as machine-generated must be routed through scans looking for hardcoded placeholder secrets, configuration credentials, and the omission of unprompted defenses (e.g., missing SSRF or path traversal validation on user-supplied parameters).

2. Guard Against the "Partial-Fix" in AI Code Assistants

If your engineering team uses automated assistants to fix code vulnerabilities flagged by security tools, never trust a simple removal of the flagged pattern. Enforce verification policies that require both the removal of the insecure signature and the presence of the correct defensive library call (e.g., checking that removing md5 actually coincides with the implementation of a strong KDF).

3. Block Hardcoded Secrets at the Commit Level

Because LLMs will routinely write self-contained runnable files containing placeholder API keys, credentials, and salts (the completeness hazard), integrate specialized secret scanning tools into git hooks. Block any commit containing patterns matching default placeholder secrets (e.g., "your-secret-key", "temp_salt", or "client_secret_placeholder").

4. Calibrate Provenance Classifiers Locally

Do not attempt to run a generic provenance classifier across your entire multi-language codebase. Section 12 shows that provenance features do not transfer cleanly across different ecosystems (falling to a near-chance 53% accuracy when transferring a Python-trained classifier to JavaScript). You must train and maintain separate, lightweight classifiers calibrated specifically to the unique scaffolding properties of each target programming language.

The Takeaway

The migration from manual Stack Overflow copy-pasting to automated LLM generation does not reduce security risk; it simply swaps the failure modes. While human developers are prone to legacy omissions and outdated standards, LLMs introduce dangerous placeholder vulnerabilities in their pursuit of functional completeness. To maintain secure systems, DevSecOps pipelines must treat code origin as a primary architectural signal, routing human-written and machine-generated artifacts to the specific security scanners designed for their respective weaknesses.

Den's Take

What strikes me most about this study isn't just the impressive 93% accuracy of the style-based random forest classifier in distinguishing human Stack Overflow snippets from LLM code. Rather, it’s the alarming 16% "partial-fix" failure rate during LLM vulnerability repairs. In production DevSecOps pipelines, a "partial fix" is often worse than no fix at all because it creates a false sense of security while leaving the underlying vulnerability exposed to exploitation.

This work on code-level stylistic fingerprinting aligns with wider observations on how black-box LLM output signatures leave distinct, lightweight stylistic artifacts in the source code they produce. However, we must stop treating automated LLM code generators as silver bullets for security patching. If nearly one in six AI-generated security repairs fails due to completeness hazards, we cannot afford to remove human-in-the-loop validation from automated software supply chains.