Data loader examples

[mbostock]

• Snowflake
• PostgreSQL
• Databricks
• BigQuery data loader example #1343
• MongoDB data loader example #1344
• MySQL data loader example #1345
• Oracle data loader example #1346
• Microsoft SQL Server data loader example #1347
• Datadog data loader example #1348
• Redshift data loader example #1385
• GitHub API
• Google Analytics API (adapt existing example)
• npm API data loader example #1349
• DuckDB loader example #1371
• Airtable data loader #1377
• US Census Bureau shapefiles example #1496
• youtube (extract from pangea)
• a parametrized list of products
  • with parametrized embedded analytics

[mbostock]

Broke out into separate issues.

[benminer]

Hey @mbostock - any update on a BigQuery data loader example? I am having some issues...

I got a static data loader working with BigQuery:

// data/bigquery.ts
import { BigQuery } from "@google-cloud/bigquery";

const bigquery = new BigQuery();

export const runQuery = async (query) => {
  const [rows] = await bigquery.query(query);
  return rows;
};

This works fine loading static data, like countries, from the database:

// data/countries.json.ts
import { runQuery } from "../bigquery";

process.stdout.write(
  JSON.stringify(
    await runQuery(`
SELECT country FROM $my-bg-table
`))
);

However, in the page I am trying to create, it needs to allow a viewer to pick a country from a dropdown. The dropdown works just fine when running, but I can't figure out how to do a dynamic query using my new bigquery.ts file. When I import in a "```js" tag, I get TypeError: Failed to resolve module specifier 'data/bigquery'. Running this with a `ts` tag simply displays the code, but doesn't run anything. Installing`@google-cloud/bigquery` via `npm i` also throws: `TypeError: Failed to resolve module specifier '@google-cloud/bigquery'` - which doesn't make sense, since it's installed. Using the `npm:` prefix throws CJS bundle errors from esbuild.

// my dashboard.md
```js
const countries = await FileAttachment("data/countries.json").json();
\```

```js
const countrySelect = Inputs.select(
  countries.map((obj) => obj.country),
  {
    label: "Select Country",
    value: "ES",
  }
);
display(countrySelect); // this works!
\```

```js
import { runQuery } from "data/bigquery"; // this throws
\```

I've tried importing the BigQuery library directly in a js block, but that leads to bundle errors from Rollup:
Failed to bundle using Rollup v2.79.1: the file imports a not supported node.js built-in module "fs".

```js
import { BigQuery } from "npm:@google-cloud/bigquery"; // throws
import { BigQuery } from "@google-cloud/bigquery"; // throws
const { BigQuery } = await import('@google-cloud/bigquery') // throws
const bigquery = new BigQuery();
const data = await bigquery.query(`
SELECT country, ymd, hour
FROM \`my-bg-table\`
WHERE ymd >= "2024-06-20"
and country= ${countrySelect}
and hour is not null
  and region is null
order by ymd, hour
`);
display(data);
\```

This makes sense, but I have a feeling I am doing something wrong here. I am new to @observable overall, so there may be something obvious here. Any help is very appreciated.

[Fil]

@benminer please open a discussion in the forum.