hash-sentinel

hash-sentinel

Check file hashes against your own threat feeds, MISP and optional public lookups, offline-first.

hash-sentinel takes MD5/SHA1/SHA256/SHA512 values, or files and directories that it hashes locally, and checks them against self-hosted feeds you control: plain hash lists, sha256sum output, CSV exports (such as MalwareBazaar's), MISP events, MISP REST search results and MISP feed directories, and URL feeds that are cached locally. Optionally it asks CIRCL hashlookup (no API key; it knows millions of good files from NSRL and flags known-malicious ones), MalwareBazaar (free abuse.ch Auth-Key) and VirusTotal (API key).

CLI, library and GitHub Action. Node.js 18+, zero runtime dependencies. Files are hashed locally and never uploaded. Online lookups are opt-in and send only the hash.

Features

• Inputs: hashes, files, directories (recursive), or stdin (-). Files are hashed with MD5, SHA1, SHA256 and SHA512 in one streaming pass, and all four are matched.
• Feed formats (auto-detected):
  • Text: one hash per line with an optional label, # comments, sha256sum/md5sum output (hash *file), BSD SHA256 (file) = hash.
  • CSV/TSV with a header: any sha256/sha1/md5/hash column, labelled from signature, malware, family, name, file_name, tags… Commented MalwareBazaar export headers (# "first_seen_utc","sha256_hash",…) are understood.
  • JSON: arrays of hashes or objects with sha256/md5/… fields.
  • MISP: single events, {"response": [...]} search results, and feed directories (manifest.json + <uuid>.json + hashes.csv). Hash attributes include md5, sha1, sha256, sha512, filename|sha256 and malware-sample, plus file-object attributes. Attributes with to_ids: true count as known malicious; others count as listed (context, not proof). Labels carry the event info, attribute comment and file name; event threat level and tags are kept.
  • URL feeds (--feed https://…) are cached in ~/.cache/hash-sentinel (or $HASH_SENTINEL_CACHE) for 6 hours (--ttl). If a download fails, the stale cache is used and a warning is reported. --offline never touches the network.
• Known-good lists (--known-good) for golden images or allow-listed tools. A hash that is both known-good and on a malicious feed raises known-good-conflict (a false positive, or a trojanised copy).
• Online sources (--online circl,malwarebazaar,virustotal): CIRCL hashlookup needs no key and returns NSRL known-good data plus a KnownMalicious flag. MalwareBazaar uses ABUSECH_AUTH_KEY (free at auth.abuse.ch). VirusTotal uses VT_API_KEY and counts 3 or more engines as malicious. Missing keys are reported as skipped, not failures. Lookups run 4 at a time with a 15 s timeout.
• -f table prints one compact line per input: malicious | listed | known-good | unknown | invalid.

Fuzzy hashes, NSRL and lookup caching

hash-sentinel ./samples --feed ssdeep-feed.txt          # flags near-duplicates (fuzzy-match)
hash-sentinel ./bin --nsrl NSRLFile.txt                  # known-good from NSRL RDS
hash-sentinel ./dl --online virustotal --rate virustotal=500 --lookup-ttl 3600

import { hashBuffer, compareSsdeep } from 'hash-sentinel';
const h = hashBuffer(buf); // { md5, sha1, sha256, sha512, ssdeep, tlsh }

Module formats and types

Ships ESM and CommonJS (import and require both work, via exports), with TypeScript declarations generated from the source at build time. Tests run on Node 18/20/22 in GitHub Actions (test/), including require/import checks against the packed tarball; releases are published with npm provenance.

Rules

Install and run

npx hash-sentinel 275a021bbfb6489e54d471899f7db9d1663fc695ec2fe2a2c4538aabf651fd0f --feed intel/hashes.txt
npx hash-sentinel ./downloads --feed misp-feed/ --feed https://intel.example.net/sha256.txt --online circl
sha256sum /usr/bin/* | cut -d' ' -f1 | npx hash-sentinel - --known-good golden.txt --online circl -f table

Config (.hash-sentinel.json) can declare feeds with options:

{
  "feeds": ["intel/blocklist.txt", { "url": "https://intel.example.net/misp/hashes.csv", "name": "corp-misp", "ttl": 3600 }, { "path": "watch.txt", "malicious": false }],
  "knownGood": ["golden-image.sha256"],
  "online": ["circl"]
}

Library

import { check, parseFeed, hashFile, lookup } from 'hash-sentinel';

const report = await check(['./suspicious.exe'], { feeds: ['misp-feed/'], online: ['circl'] });
report.targets[0].data.status;   // 'malicious' | 'listed' | 'known-good' | 'unknown'
report.targets[0].data.matches;  // [{ feed, label, malicious, hash, meta }]

Options

Full CLI reference (hash-sentinel --help):

hash-sentinel 1.1.0 — look up file hashes in self-hosted feeds and optional public sources

Usage: hash-sentinel <hash|file|dir|-> [...] [options]

Inputs are MD5/SHA1/SHA256/SHA512 values, files (hashed locally, never uploaded) or directories.
"-" reads hashes or paths from stdin, one per line.

Feeds (offline, self-hosted)
      --feed <src>           Hash list, CSV (e.g. MalwareBazaar export), MISP event JSON, MISP feed
                             directory, or an http(s) URL (cached; repeatable or comma-separated)
      --known-good <src>     Known-good hash list(s) (e.g. your golden images)
      --nsrl <src>           NSRL RDS CSV/hash list (file or URL), treated as known-good
      --no-fuzzy             Skip ssdeep/TLSH fuzzy hashing of files
      --ssdeep-threshold <n> Minimum ssdeep similarity score for fuzzy-match (default 60)
      --tlsh-threshold <n>   Maximum TLSH distance for fuzzy-match (default 50)
      --lookup-ttl <s>       Cache online lookup results for s seconds (default 86400; 0 disables)
      --rate <src=n>         Requests per minute for an online source (default virustotal=4, circl=120, malwarebazaar=60)
      --ttl <seconds>        URL feed cache lifetime (default 21600)
      --offline              Never touch the network (cached URL feeds only)
Online (opt-in; only the hash is sent)
      --online <list>        circl, malwarebazaar, virustotal
                               circl: CIRCL hashlookup, no key (NSRL known-good + KnownMalicious)
                               malwarebazaar: needs ABUSECH_AUTH_KEY (free at auth.abuse.ch)
                               virustotal: needs VT_API_KEY
  -f table                   One line per input: status, target, label
Policy and output
  -c, --config <file>        JSON config (default .hash-sentinel.json if present)
  -f, --format <fmt>         text (default) | json | sarif | markdown
      --json/--sarif/--markdown <file>  Also write that format to a file
      --fail-on <sev>        critical | high | medium | low | info | none (default high)
      --list-rules           Print rules and default severities
      --no-color             Disable colours
  -v, --version / -h, --help

Exit codes: 0 = passed, 1 = findings at or above --fail-on, 2 = usage error or a target could not be checked.

Configuration

Put a .hash-sentinel.json in the working directory (or pass --config <file>). Every key is optional; CLI flags override the file.

{
  "failOn": "high",
  "severity": { "known-malicious": "high" },
  "allow": [
    { "rule": "known-malicious", "match": "e3b0c442*", "reason": "Accepted risk, ticket SEC-123", "expires": "2027-03-31" }
  ]
}

• severity changes the severity of any rule, or turns it off.
• allow suppresses findings. Each entry needs a rule (globs such as known-* work) and a reason, and can be narrowed with target and match (a glob tested against the target, location and value). Suppressed findings are listed in the JSON report. When expires (YYYY-MM-DD) has passed, the finding comes back together with an allow-expired finding, so exceptions can't be forgotten. Entries that match nothing are reported as allow-unused (info).

Output and exit codes

• --format text|json|sarif|markdown for stdout, plus --json, --sarif and --markdown <file> to write extra copies in one run. SARIF 2.1.0 works with GitHub code scanning.
• --fail-on critical|high|medium|low|info|none (default high).
• Exit code 0: passed. 1: at least one finding at or above --fail-on. 2: usage or config error, or a target could not be checked.
• --list-rules prints every rule with its default severity.

GitHub Action

permissions:
  contents: read
  security-events: write   # only for the SARIF upload
steps:
  - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
    with: { persist-credentials: false }
  - uses: MrLouisWright/hash-sentinel@v1
    with:
      targets: 'downloads/'
      fail-on: high
      sarif: hash-sentinel.sarif
  - uses: github/codeql-action/upload-sarif@v3
    if: always()
    with: { sarif_file: hash-sentinel.sarif }

Inputs: targets, fail-on, config, sarif, json, job-summary (Markdown report in the job summary, default true), args (extra CLI flags), feeds, online. Outputs: markdown-file, exit-code.

Requirements

Node.js 18 or newer. No runtime dependencies.

Support

If this project is useful to you, you can support its development with a donation: Buy me a coffee or GitHub Sponsors. It helps keep these tools maintained, documented and free for everyone.

Credit (optional)

Free to use under MIT, no attribution required. If hash-sentinel helps you, a link back is appreciated:

<a href="https://louiswright.co.uk">Built with hash-sentinel by Louis Wright</a>

[![Built with hash-sentinel by Louis Wright](https://img.shields.io/badge/built%20with-hash--sentinel-blue)](https://louiswright.co.uk)

License

MIT © Louis Wright (Louis Wright)