Which data sources are supported by the rfp-ingest skill?

The skill currently supports SAM.gov (REST API), Maryland eMMA (Web scraping), and RFPMart (both REST API and manual CSV uploads).

How does the skill handle duplicate RFP records?

It employs a three-tier deduplication strategy: exact ID/Source matching, fuzzy title similarity within deadline windows, and URL canonicalization.

Can I filter RFPs for specific industries like IT?

Yes, the skill includes built-in mapping for IT-relevant prefixes such as SW (Software), ITES (IT Services), and NET (Networking) to prioritize relevant leads.

Does this skill support automated data updates?

Yes, it includes implementation patterns for Convex cron jobs that can be configured to poll APIs like SAM.gov at regular intervals (e.g., every 6 hours).

How does it detect if a contract requires a security clearance?

The ingestion logic includes an eligibility pre-filtering engine that uses regex patterns to flag disqualifiers or requirements like 'security clearance' or 'US citizenship' automatically.

RFP Data Ingestion Engine

Name: RFP Data Ingestion Engine
Author: Atemndobs

byAtemndobs

Web Scraping & Data Collection

Automates the collection, normalization, and deduplication of Request for Proposal (RFP) opportunities from government and private data sources.

About

The rfp-ingest skill provides a robust framework for centralizing RFP discovery by aggregating data from SAM.gov, Maryland eMMA, and RFPMart into a unified canonical schema. It manages the complexities of API rate limiting, web scraping, and manual CSV parsing while implementing sophisticated deduplication logic and eligibility pre-filtering. Designed for developers building procurement platforms, it includes production-ready Convex actions, schema definitions, and automated cron job configurations to ensure a reliable and searchable database of contract opportunities.

Key Features

Multi-source aggregation from SAM.gov, eMMA, and RFPMart
Canonical schema normalization for consistent data processing
Automated eligibility detection for security clearances and set-asides
Fuzzy-match deduplication and URL canonicalization
Production-ready Convex backend integration and scheduled crons
0 GitHub stars

Use Cases

Building a centralized monitoring dashboard for government contract leads
Filtering massive RFP feeds for specific IT-relevant NAICS codes
Automating the synchronization of manual CSV email alerts into a database

About

Key Features

Multi-source aggregation from SAM.gov, eMMA, and RFPMart
Canonical schema normalization for consistent data processing
Automated eligibility detection for security clearances and set-asides
Fuzzy-match deduplication and URL canonicalization
Production-ready Convex backend integration and scheduled crons
0 GitHub stars

Use Cases

Building a centralized monitoring dashboard for government contract leads
Filtering massive RFP feeds for specific IT-relevant NAICS codes
Automating the synchronization of manual CSV email alerts into a database