How do I handle model serialization?

The skill provides a decision path for choosing between JSON (XGBoost), Pickle (Sklearn), state_dict (PyTorch), or save_pretrained (HuggingFace) based on the algorithm.

Does this skill help with adding custom algorithms?

Yes, it provides a detailed checklist for creating custom nn.Modules and wrapping them in the Tuner and Instance layers for full pipeline compatibility.

When should I use the haipipe-nn-0-overview skill?

This should be read first before any other haipipe-nn skill to understand the framework patterns, decision logic, and naming conventions required for the pipeline.

What is the 4-layer architecture in HAIPipe?

The architecture separates concerns into Layer 1 (Algorithm/Library), Layer 2 (Tuner/Wrapper), Layer 3 (Instance/Orchestrator), and Layer 4 (ModelSet/Pipeline) to ensure modularity and scalability.

What data formats are supported by the decision tree?

The guide helps categorize algorithms into Nixtla DataFrames, Sparse matrices, PyTorch tensors, HuggingFace Datasets, or LLM API formats.

HAIPipe NN Pipeline Overview

Name: HAIPipe NN Pipeline Overview
Author: jluo41

byjluo41

0•

Data Science & ML

Provides a foundational architecture map and decision guide for managing neural network pipelines within the HAIPipe research framework.

This skill serves as the primary entry point for the HAIPipe neural network ecosystem, defining a strict 4-layer architectural separation between raw algorithms, tuners, model instances, and pipeline orchestration. It helps researchers and developers integrate new machine learning models by providing structured decision trees for data formats, step-by-step implementation checklists, and standardized YAML configuration templates. By centralizing the model registry and framework patterns, it ensures consistent implementation across various model families including time-series forecasting, treatment effect modeling, and foundation models.

Key Features

010 GitHub stars

02Comprehensive Step-by-Step New Model Integration Checklist

03Algorithm Decision Tree for data formats, serialization, and Optuna tuning

04Dynamic Model Registry Management and Registry Snapshots

054-Layer Architecture Mapping (Algorithm, Tuner, Instance, ModelSet)

06Standardized YAML Configuration Templates for diverse ML tasks

Use Cases

01Integrating external libraries like HuggingFace, XGBoost, or Nixtla into a unified training workflow.

02Architecting a standardized neural network pipeline for academic research and paper incubation.

03Configuring complex model instances for specialized tasks like time-series forecasting or treatment effect estimation.

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add jluo41/research-skills haipipe-nn-0-overview

For use in Claude.ai and ChatGPT

Download Skill