How does this skill handle Knack's API limits?

It automatically detects the total number of pages and iterates through them, incorporating a 100ms delay between requests to stay within the 10 requests per second limit.

Is this compatible with other Knack-related skills?

Yes, it is designed to work alongside tools like knack_reader for base operations and knack_cache_optimizer for performance.

Does it support custom filtering and sorting?

Absolutely. Parameters for filters and sort orders can be passed to the core functions to ensure consistent data views across all retrieved pages.

What happens if a request fails during a long fetch?

The skill implements an error recovery system with a default of 3 retries and exponential backoff, specifically designed to handle 429 Rate Limit errors.

Can I process data without loading everything into memory?

Yes, the skill includes a stream_records function specifically designed for memory-efficient processing of large datasets by handling one batch at a time.

Knack API Pagination Manager

Name: Knack API Pagination Manager
Author: willsigmon

bywillsigmon

•

Database Management

Automates full dataset retrieval from the Knack API by managing page limits, rate limiting, and record streaming.

This skill provides specialized logic for interacting with the Knack database API, overcoming the default 1,000-record-per-page limit through automated multi-page navigation. It features robust error handling with exponential backoff, rate limit protection to prevent 429 errors, and memory-efficient streaming for processing massive datasets without exhausting system resources. It is essential for developers managing thousands of records who require comprehensive reporting, bulk data synchronization, and reliable data integrity across large Knack objects.

Key Features

017 GitHub stars

02Automatic retry logic with exponential backoff for API errors

03Memory-efficient record streaming for large-scale processing

04Built-in rate limit protection with configurable delays

05Real-time progress tracking for long-running data fetches

06Automated multi-page data retrieval beyond the 1,000-record limit

Use Cases

01Syncing complete Knack object data to external databases or compliance tools

02Generating comprehensive quarterly reports for 3,500+ record inventories

03Performing bulk data analysis and county-level breakdowns across entire datasets

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add willsigmon/sigstack knack-pagination

For use in Claude.ai and ChatGPT

Download Skill