Repository avatar
Search & Data Extraction
v1.0.0
active

cleaningredients

io.github.burmanharry/cleaningredients

Research China-origin botanicals, supplier evidence gaps, sourcing briefs and ingredient costs.

Documentation

CleanIngredients

Public directory of plant-extract manufacturers for US supplement brands. Scores measure documented transparency. They are not safety or quality ratings.

Scripts

npm install
npm run dev
npm test
npm run typecheck
npm run lint
npm run build

Open http://localhost:3000.

npm run build validates the directory catalog before next build.

Inquiries

Sample, quote, correction, and sourcing-inquiry forms save to the Postgres table buyer_requests when DATABASE_URL is set. Apply deploy/postgres/001_buyer_requests.sql, then deploy/postgres/002_sourcing_inquiries.sql, with psql on the dedicated database. Sample, quote, and correction inserts do not use the columns added in 002. A sourcing inquiry does not succeed until 002 is applied. If DATABASE_URL or the table is missing, the form reports that nothing was saved.

There is no email when a row is saved. An operator reviews sourcing inquiries by reading buyer_requests where request_type = 'inquiry'. The public site has no page or API that lists those rows. Steps are in deploy/hetzner.md.

Optional rate-limit backend: UPSTASH_REDIS_REST_URL and UPSTASH_REDIS_REST_TOKEN. Without them, the server keeps a short in-memory limit.

Public directory content lives in content/directory. Dev scoring fixtures live in tests/ and are not loaded by the site.

Ingredient trends

/trends charts US demand signals for each ingredient from content/trends/snapshot.json. Refresh it with:

NODE_USE_ENV_PROXY=1 npm run trends:update

NODE_USE_ENV_PROXY=1 is only needed behind an HTTPS proxy. Pass -- --source=google or -- --source=keepa to refresh one source.

  • Google Trends needs no key. Terms live in content/trends/terms.ts.
  • Keepa needs KEEPA_API_KEY in the environment or .env.local. Each refresh uses roughly 30 tokens per ingredient. The script waits when tokens run out.

Locally, the refresh writes the committed content/trends/snapshot.json; commit it to publish. On the VPS, a monthly cron job writes data/trends/snapshot.json instead, and the running site shows it on the next page load. See deploy/hetzner.md.

Hosting

The VPS layout is Docker Compose on localhost port 3004, behind Cloudflare Tunnel. Steps are in deploy/hetzner.md. Inquiry variables stay in the server .env and are not baked into the image.

Free public reference data

Ingredient pages show PubChem reference compounds and openFDA food-recall keyword counts. Run npm run data:refresh to refresh the source-linked snapshot. These automated references do not change supplier scores or verify safety, origin, or manufacturing. FDA Dashboard access remains pending credentials. Deployment and daily refresh instructions are in deploy/external-data.md.

No installation packages available.