Skip to content

Dockerfile/Gemfile/Makefile/Rakefile/LICENSE and other extensionless, non-shebang project files currently produce no visible trace in graphify's output — not classified, not counted anywhere #1692

Description

@sub4biz

Summary

The main thing this issue is about: right now, when classify_file() returns None for a file, that
file leaves no trace anywhere in graphify's own reporting — not in total_files, not as a skipped count,
nothing. This happens for any extensionless file without a shebang line — in practice, mostly declarative
project files like Dockerfile, Gemfile, Makefile, Rakefile, and LICENSE. A user has no way to
discover, from graphify's own output, that these files were ever considered.

Separately, and more as a "while we're looking at this" thought than a core ask: a couple of these
filenames look like they'd fit naturally into flows the tool already has, which is covered further down.

Evidence

Real counts from one monorepo's work/ tree (find ... -type f ! -name "*.*", excluding .git/ and
graphify-out/), grouped by basename:

26 Dockerfile
21 Gemfile
15 Rakefile
15 Makefile
 6 LICENSE
 6 rake / 6 rails / 6 bundle / 6 spring / 5 pre-commit / 4 update / 4 console / 3 yarn / 3 Capfile /
 3 Berksfile / 2 start / 2 sbt / 2 gradlew / ... (various project scripts and git hooks)

None of these carry a shebang line (Dockerfile/Gemfile/Makefile/Rakefile/LICENSE are
declarative/data files, not executables) — some of the scripts further down the list (rake, console,
pre-commit) may have shebangs and would be covered by #1680 once merged, but the top four alone
account for 83 files in this one corpus.

Where this happens

detect.py:386-420 (classify_file):

def classify_file(path: Path) -> FileType | None:
    from graphify.manifest_ingest import is_package_manifest_path
    if is_package_manifest_path(path):
        return FileType.CODE
    if path.name.lower().endswith(".blade.php"):
        return FileType.CODE
    ext = path.suffix.lower()
    if not ext:
        return _shebang_file_type(path)     # the one path currently in place for extensionless files
    if ext in CODE_EXTENSIONS:
        return FileType.CODE
    ...
    return None

_shebang_file_type(path) (detect.py:~370-383) reads the first line and returns FileType.CODE only
if it recognizes the interpreter in _SHEBANG_CODE_INTERPRETERS; otherwise it returns None. There isn't
currently a path for "extensionless, no shebang, but a well-known filename" — the manifest check
(is_package_manifest_path) is the closest existing analogue, since it already runs before the
extension check specifically to catch pyproject.toml/go.mod/pom.xml/apm.yml by exact filename
regardless of what a suffix lookup would say (manifest_ingest.py:28-34):

PACKAGE_MANIFEST_NAMES: dict[str, str] = {
    "apm.yml": "apm",
    "apm.yaml": "apm",
    "pyproject.toml": "python",
    "go.mod": "go",
    "pom.xml": "maven",
}

Impact

Core concern — discoverability. This is a quieter gap than the sibling "no tree-sitter grammar for
this extension" case (see the AST-progress-line issue filed alongside this one), because that case at
least shows up as a number in total_files vs. uncached_work. A classify_file() -> None file doesn't
enter total_files, docs, or any other bucket the detect summary reports — there's currently no way to
tell, from the tool's own output, that these files were even looked at. This seems like it would be
useful to have regardless of what happens with the rest of this issue, and feels consistent with how
transparent the rest of the pipeline already is elsewhere (e.g. the extraction cost/token summary, or the
EXTRACTED/INFERRED/AMBIGUOUS confidence labels) — a "N files did not match any known type" line would
just extend that same visibility to this one stage.

A secondary thought — these two specific filename families, if useful:

  • Gemfile looks like the same shape of entity PACKAGE_MANIFEST_NAMES already exists to handle — a
    dependency manifest, same as pyproject.toml/go.mod/pom.xml, just missing from the table by
    filename rather than by design. If the table's purpose is "recognize dependency manifests regardless of
    extension," Gemfile seems like a natural fit for the existing mechanism rather than a new one. (Not
    Gemfile.lock — that one's already deliberately handled, it's in _SKIP_FILES alongside
    package-lock.json/yarn.lock/poetry.lock as a generated lockfile, which looks like the right call
    and isn't part of what this issue is about.)

  • Dockerfile/Makefile/Rakefile are a different shape — not a dependency manifest so much as a
    build/deploy description — but they might be worth a look specifically because of graphify's cross-repo
    merge / monorepo flow. A FROM base-image:tag line in one service's Dockerfile and the same base image
    in another service's Dockerfile is exactly the kind of connection that wouldn't show up from source-code
    imports alone (the two services don't reference each other in code), but would be a real architectural
    relationship in a monorepo graph — shared base images, shared build/task dependencies across
    repos/services. Since surfacing that kind of cross-component connection seems to be part of what the
    monorepo/cross-repo merge feature is for, these two filenames might be worth considering as a next step
    after Gemfile — no strong opinion on priority, just flagging the scenario since it lines up with a
    capability the tool already advertises.

LICENSE is a harder case — not code or a manifest, arguably closer to FileType.DOCUMENT (prose
content). If the discoverability fix below lands, LICENSE files would at least surface as
"unclassified" rather than leaving no trace, which seems like a reasonable place to leave it pending any
decision on what a LICENSE node would even represent in a knowledge graph.

Possible next steps

  1. Surface a "classified as nothing" count in detect's summary output. This seems like the more
    self-contained, low-risk piece, independent of anything else in this issue — even a generic
    "N files did not match any known type (run with --verbose to list)" line would make this stage
    discoverable without needing to read detect.py source to find out it happened.
  2. If it seems worthwhile: add Gemfile to PACKAGE_MANIFEST_NAMES — same mechanism already used for
    pyproject.toml/go.mod/pom.xml. (Gemfile.lock doesn't need anything — it's already correctly
    routed to _SKIP_FILES.)
  3. If it seems worthwhile and in scope: consider Dockerfile/Makefile/Rakefile (and optionally
    Vagrantfile, Berksfile, Capfile, Podfile, Brewfile — same extensionless-declarative-file
    family) for their own lightweight extractor, given the cross-repo/monorepo scenario above. Since these
    aren't dependency-list-shaped the way PACKAGE_MANIFEST_NAMES entries are, this would probably want its
    own extract_dockerfile/extract_makefile rather than reusing extract_package_manifest's output
    shape — happy to sketch what that might look like if it's a direction worth pursuing.

Relationship to PR #1680

Not a duplicate. #1680 fixes shebang-bearing extensionless scripts by adding _SHEBANG_DISPATCH in
extract.py's _get_extractor — that's the AST-extractor-selection stage, and only ever engages for
files _shebang_file_type already classified as CODE. This issue is one stage earlier
(classify_file itself returning None) and covers files that have no shebang to detect in the first
place. Both fixes are independently useful for extensionless-file coverage; neither supersedes the other.

Status check (2026-07-06)

0.9.7 release notes list "Extensionless shebang CLIs (like devctl, manage) are now extracted instead
of being silently dropped" — this is the #1680 fix (shebang-bearing files), already addressed as a
separate, non-overlapping case in the "Relationship to PR #1680" section above. Re-verified directly
against the installed 0.9.7 source (not just release notes): classify_file() in detect.py:386-420 and
PACKAGE_MANIFEST_NAMES in manifest_ingest.py:28-34 are unchanged from what's quoted above — line
numbers and logic match exactly. Dockerfile/Gemfile/Makefile/Rakefile/LICENSE still fall through
to None in 0.9.7. Also checked _SKIP_FILES (detect.py:691-695) while re-verifying — Gemfile.lock
is already there by design (see the note under "A secondary thought" above); nothing else in that list
overlaps with what's discussed here.

Environment

  • Original observation: graphifyy 0.9.6, on a large polyglot monorepo (work/, ~20.6k code files) during
    a full rebuild; counts above are from a direct filesystem scan (find ... -type f ! -name "*.*"), not
    from graphify's own output — which is itself the thing being reported here (there's currently no counter
    that would have surfaced these files at all).
  • Re-verified against graphifyy 0.9.7 (2026-07-06) by reading the installed package source directly —
    classify_file(), PACKAGE_MANIFEST_NAMES, and _SKIP_FILES all match what's quoted above, so the
    behavior described is current, not a stale observation from an older version.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions