data-scrape-api
Installation
Entrypoint
Types
DataScrapeApi

data-scrape-api

CI for data-only APIs that scrape upstream → commit JSON → serve via
GH raw / jsdelivr (oriz-india-holidays, oriz-air-quality, oriz-currency-rates,
oriz-flow-fii-dii, oriz-gold-silver, oriz-ifsc, india-budget,
india-petrol-diesel, india-train-schedules).

No Astro build. Pipeline:
1. lint scrape script (biome or ruff via --if-present)
2. run scraper in sandbox
3. validate output JSON parses
4. staleness gate — fail if output hasn't changed vs prior commit
AND last-modified-of-source-of-truth is older than 7d
(skipped when running in CI without a real scrape target)

Installation

dagger install github.com/chirag127/workflows/dagger/data-scrape-api@4fa20ea6e7332d1b89d2dfaea1bd41fce8c4941d

Entrypoint

Return Type

DataScrapeApi

Example

dagger -m github.com/chirag127/workflows/dagger/data-scrape-api@4fa20ea6e7332d1b89d2dfaea1bd41fce8c4941d call \

func (m *MyModule) Example() *dagger.DataScrapeApi  {
	return dag.
			Datascrapeapi()
}

@function
def example() -> dagger.DataScrapeApi:
	return (
		dag.data_scrape_api()
	)

@func()
example(): DataScrapeApi {
	return dag
		.dataScrapeApi()
}

Types

DataScrapeApi 🔗

lint() 🔗

Return Type

String !

Arguments

Name	Type	Default Value	Description
source	Directory !	-	No description provided

Example

dagger -m github.com/chirag127/workflows/dagger/data-scrape-api@4fa20ea6e7332d1b89d2dfaea1bd41fce8c4941d call \
 lint --source DIR_PATH

func (m *MyModule) Example(ctx context.Context, source *dagger.Directory) string  {
	return dag.
			Datascrapeapi().
			Lint(ctx, source)
}

@function
async def example(source: dagger.Directory) -> str:
	return await (
		dag.data_scrape_api()
		.lint(source)
	)

@func()
async example(source: Directory): Promise<string> {
	return dag
		.dataScrapeApi()
		.lint(source)
}

jsonValidate() 🔗

Return Type

String !

Arguments

Name	Type	Default Value	Description
source	Directory !	-	No description provided

Example

dagger -m github.com/chirag127/workflows/dagger/data-scrape-api@4fa20ea6e7332d1b89d2dfaea1bd41fce8c4941d call \
 json-validate --source DIR_PATH

func (m *MyModule) Example(ctx context.Context, source *dagger.Directory) string  {
	return dag.
			Datascrapeapi().
			Jsonvalidate(ctx, source)
}

@function
async def example(source: dagger.Directory) -> str:
	return await (
		dag.data_scrape_api()
		.jsonvalidate(source)
	)

@func()
async example(source: Directory): Promise<string> {
	return dag
		.dataScrapeApi()
		.jsonValidate(source)
}

ci() 🔗

Full CI: lint (advisory) + JSON validation (gate). Scrape execution is left to the repo’s own workflow — this module doesn’t shell out to arbitrary scraper code by default.

Return Type

String !

Arguments

Name	Type	Default Value	Description
source	Directory !	-	No description provided

Example

dagger -m github.com/chirag127/workflows/dagger/data-scrape-api@4fa20ea6e7332d1b89d2dfaea1bd41fce8c4941d call \
 ci --source DIR_PATH

func (m *MyModule) Example(ctx context.Context, source *dagger.Directory) string  {
	return dag.
			Datascrapeapi().
			Ci(ctx, source)
}

@function
async def example(source: dagger.Directory) -> str:
	return await (
		dag.data_scrape_api()
		.ci(source)
	)

@func()
async example(source: Directory): Promise<string> {
	return dag
		.dataScrapeApi()
		.ci(source)
}