Dupehound: Offline and Deterministic Code Duplicate Detector for Agentic Codebases

AST-Based Structural Fingerprinting

Unlike text-based search engines, dupehound drops comments, replaces identifiers, strings, and numbers with sentinels, and analyzes the underlying abstract syntax tree. It uses k-grams of 10 tokens with rolling hashes and robust winnowing to guarantee that any shared sequence of 17 normalized tokens is caught. Similarity is calculated using exact Jaccard index, and matching clusters are generated via union-find.

CLI Commands and Integration

Dupehound provides three core commands:

• dupehound scan [path] scans a directory, ranks duplicate clusters by deletable lines, and outputs a 'slop score' representing the percentage of redundant code.
• dupehound history reads git blobs directly from the object database without checking out files, mapping out exactly when duplication spiked over time.
• dupehound check operates as a CI gate or pre-commit hook. It indexes the codebase at the base git revision and analyzes only the newly added or modified functions, exiting with code 1 upon discovering duplicates.

Prompting Agents for Code Reuse

To prevent agents from generating duplicate code, developers can pipeline the output of dupehound check directly to their agent. Placing the output or guidelines within a CLAUDE.md or AGENTS.md file forces the agent to inspect the existing original function and refactor its code to reuse it rather than committing new redundant blocks.