aidp-ai-sql

First-run setup for the Snowflake to AIDP migrator. Finds or creates the one migration config file (snowmig-config.yaml), reads every field back with secrets masked, verifies Snowflake authentication by password, key-pair, programmatic access token or SSO, verifies the AIDP end, then smoke-tests the connection and reports the account, region, role and warehouse. Use the first time the migrator runs on a machine, or when any other stage fails with an authentication, connection or "no config found" error.

Build a high-level Snowflake to AIDP migration plan and present it for approval. States which objects can migrate and which cannot with a brief reason for each, applies user-supplied restrictions such as excluded databases or size caps, derives dependency order so views follow their base tables, and lists the Silver and Gold job stubs. Use after an estate assessment, or when the user asks what can be migrated, in what order, why something is excluded, or wants to see the migration plan.

Create the medallion architecture on Oracle AI Data Platform - registering an EXTERNAL catalog of source type SNOWFLAKE by default, and generating Spark SQL for schemas, tables and views from an approved Snowflake plan only when the user has explicitly asked for a Standard catalog. This stage copies no data - rows are copied only by the snowmig_02_copy_schema job, when the operator runs it. Use when the user asks to create the medallion structure, soft clone, shallow clone, create the target catalog, or deploy the target schema.

Map Snowflake warehouses onto AIDP Spark compute clusters and produce a sizing and cost proposal. Inventories every warehouse with its size, multi-cluster scaling policy and auto-suspend, pulls observed credit consumption from ACCOUNT_USAGE where permitted, and proposes worker counts and autoscale ceilings per warehouse. Use when the user asks about compute sizing, warehouse equivalence, cluster shapes, credits, or what the migration will cost.

Generate the table-creation script for a Standard AIDP catalog as an executable notebook, locally and offline. --upload is a dry run and is refused with --execute; the structure itself is created by the snowmig_01_structure workflow at S10. The script creates schemas, then tables, then views in dependency order, prints per-object progress with elapsed time so a long run stays visible, and verifies each object individually at the end. This script creates structure only and copies no data - every table arrives with zero rows; rows are copied only by the snowmig_02_copy_schema job, when the operator runs it. Use when the user has explicitly asked for a Standard catalog, or wants the migration delivered as a runnable script rather than executed straight from the CLI.

Read-only PREVIEW of a Snowflake environment from the operator's machine - inventory of tables and views with row counts, byte sizes and column types, plus the census of objects that are not tables or views, table-maintenance state and security posture. Use ONLY when the user wants to look at an account without migrating it - answering what is in there, how big the tables are, what the security posture looks like. This is NOT the discovery step of a migration: a migration discovers inside AIDP as a workflow (runbook S6), because a laptop-side read leaves no log and no evidence on the platform. If the user asked to migrate, route to snowflake-migrator-overview and follow S1 through S12.

Human approval gate over LOW/MEDIUM/MANUAL-confidence conversions via `infa2aidp cli review generate|import`. Use after infa-migrate-mapping, before trusting any generated notebook, to surface every low-confidence conversion and REVIEW REQUIRED marker for a human decision (approve/edit/reject) rather than letting them ship silently.

Compare source-DB rows against a migrated AIDP target via `infa2aidp cli reconcile`, driven by a YAML config (row_count / schema / aggregate / data checks, with ignore_columns for audit/surrogate columns that never match). Use after a migrated notebook has actually run and populated its AIDP target — this is the customer-facing proof the migration is correct, not merely that it ran.

About 450 ready-made HyperFrames blocks and components installed as editable source - transitions, overlays, titles, kinetic type, captions, charts, map overlays, 3D, camera moves. Use it before hand-building any named visual (CRT scanlines, glitch, film grain, a chart); also for hyperframes add and catalog.

Safely migrate the schema + backfill data on a deployed Convex app with @convex-dev/migrations. TRIGGER when changing the schema on a live app, backfilling data, or after a schema-validation error from a change. SKIP a fresh local app.

Seed the Convex database with starter/fixture data or import an existing dataset (CSV/JSON). TRIGGER when the user wants sample/seed data or to import a dataset into Convex. Idempotent, schema-matching.

Runs SQL analytics on SageMaker Catalog asset metadata tables exported as Apache Iceberg in S3 Tables. Covers governance queries, asset growth tracking, ownership audits, time-travel over catalog state, and metadata quality analysis. Applies when querying catalog inventory, finding assets without descriptions, comparing catalog snapshots, or auditing data ownership. Trigger phrases: catalog inventory SQL, how many assets, assets without descriptions, asset growth over time, who owns this data, catalog governance, data quality audit, catalog analytics.

Handles the full DMS Schema Conversion lifecycle including creating migration projects, converting database schemas to a target engine, running compatibility assessments, navigating metadata trees, exporting converted DDL to S3, applying schema changes to a target database, and converting SQL statements between database engines. Applies when migrating database schemas between heterogeneous engines using AWS DMS Schema Conversion.

Provides authoritative guidance on Amazon Neptune Database and Neptune Analytics for graph, knowledge-graph, and relationship-heavy workloads — fraud detection / fraud rings, agentic memory / chatbot context across sessions, recommendations, identity resolution, Gremlin / openCypher / SPARQL queries, supernode / slow traversal, Neo4j to Neptune migration / APOC compatibility, Neptune Database vs Analytics engine selection, PageRank / community detection, GraphRAG, and connectivity from Lambda / EC2 / applications. Creates and modifies Neptune Database clusters/instances and Neptune Analytics graphs on explicit user confirmation; blocks destructive operations (delete, reset-graph, failover, major upgrade) and redirects to change-control.

Designs, reviews, and debugs DynamoDB data layers from design axioms — enumerates access patterns, chooses partition/sort keys and GSIs, decides single-table vs. multi-table, configures Streams, Global Tables, TTL, vector indexes for similarity search, and zero-ETL integrations to OpenSearch/Redshift/SageMaker Lakehouse, and produces a defensible data-layer design with a monthly cost estimate and optional live validation. Applies whenever a user is designing, reviewing, or refactoring anything backed by DynamoDB — schemas, access patterns, GSIs, single- vs. multi-table choices, Streams consumers, transactional outboxes, Global Tables, zero-ETL pipelines, or storing embeddings and running semantic/vector similarity search with SearchVectors on items already in DynamoDB — even when they don't say "axioms" or "design review." Also applies when debugging hot partitions, throttling, unbounded Scans, LWW conflicts, or surprise bills on DynamoDB workloads.

Exports Amazon RDS or Aurora database snapshots to Amazon S3 in Apache Parquet format for analytics, backup, or data migration. Handles snapshot selection or creation, IAM role setup, KMS encryption, S3 bucket preparation, export task execution, progress monitoring, and data verification. Use when exporting RDS/Aurora data to S3 for Athena, Glue, or Redshift Spectrum consumption.