Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FLAG-1263: Data mart api client #4966

Draft
wants to merge 4 commits into
base: develop
Choose a base branch
from
Draft
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
create api routes
wri7tno committed Mar 7, 2025
commit 29ccaadce324e2afefc1fffbd2ac9ed80ac8b8d1
118 changes: 118 additions & 0 deletions pages/api/datamart/v0/land/[...slug].js
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
// eslint-disable-next-line no-unused-vars
import { NextApiRequest, NextApiResponse } from 'next';
import {
createRequestByGeostoryId,
getDataByGeostoreId,
getDataFromLink,
} from 'services/datamart';
import { GFW_DATA_API, GFW_STAGING_DATA_API } from 'utils/apis';

// types
/**
* @typedef {object} DataLinkObject
* @property {string} link - The URL to POST the content.
*/

/**
* @typedef {object} GetResponseObject
* @property {string} status - status.
* @property {DataLinkObject} data - data link object.
*/

/**
* @typedef {object} NotFoundObject
* @property {string} status - status.
* @property {string} message - message.
*/
// END types

const ENVIRONMENT = process.env.NEXT_PUBLIC_FEATURE_ENV;

// We never use the staging api
export const DATA_API_URL =
ENVIRONMENT === 'staging' ? GFW_STAGING_DATA_API : GFW_DATA_API;

/**
* @param {NextApiRequest} req
* @param {NextApiResponse} res
*/
const fetchDataByDatasetAndGeostore = async (req, res) => {
const { query } = req;
// TODO: add more parameters to the query like, global, adm9, adm1, etc etc etc
const { slug: slugs, geostore_id, canopy_cover } = query;

if (slugs.length === 0) {
res.status(400).send();
return;
}

if (slugs.length === 1) {

Check failure

Code scanning / CodeQL

Type confusion through parameter tampering Critical

Potential type confusion as
this HTTP request parameter
may be either an array or a string.

Copilot Autofix AI 10 days ago

To fix the problem, we need to ensure that the slugs parameter is always treated as an array of strings. We can do this by checking the type of slugs and converting it to an array if it is not already one. This will prevent type confusion attacks and ensure that the code behaves as expected.

We will modify the fetchDataByDatasetAndGeostore and postData functions to include type checks and conversions for the slugs parameter. Specifically, we will:

  1. Check if slugs is an array. If not, convert it to an array containing the single value.
  2. Proceed with the existing logic, which assumes slugs is an array.
Suggested changeset 1
pages/api/datamart/v0/land/[...slug].js

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/pages/api/datamart/v0/land/[...slug].js b/pages/api/datamart/v0/land/[...slug].js
--- a/pages/api/datamart/v0/land/[...slug].js
+++ b/pages/api/datamart/v0/land/[...slug].js
@@ -40,3 +40,4 @@
   // TODO: add more parameters to the query like, global, adm9, adm1, etc etc etc
-  const { slug: slugs, geostore_id, canopy_cover } = query;
+  const { slug: slugsParam, geostore_id, canopy_cover } = query;
+  const slugs = Array.isArray(slugsParam) ? slugsParam : [slugsParam];
 
@@ -78,3 +79,4 @@
   // TODO: add more parameters to the query like, global, adm9, adm1, etc etc etc
-  const { slug: slugs, geostore_id, canopy_cover } = query;
+  const { slug: slugsParam, geostore_id, canopy_cover } = query;
+  const slugs = Array.isArray(slugsParam) ? slugsParam : [slugsParam];
 
EOF
@@ -40,3 +40,4 @@
// TODO: add more parameters to the query like, global, adm9, adm1, etc etc etc
const { slug: slugs, geostore_id, canopy_cover } = query;
const { slug: slugsParam, geostore_id, canopy_cover } = query;
const slugs = Array.isArray(slugsParam) ? slugsParam : [slugsParam];

@@ -78,3 +79,4 @@
// TODO: add more parameters to the query like, global, adm9, adm1, etc etc etc
const { slug: slugs, geostore_id, canopy_cover } = query;
const { slug: slugsParam, geostore_id, canopy_cover } = query;
const slugs = Array.isArray(slugsParam) ? slugsParam : [slugsParam];

Copilot is powered by AI and may make mistakes. Always verify output.
const dataByGeostore = await getDataByGeostoreId({
dataset: slugs[0],
geostoreId: geostore_id,
canopy: canopy_cover,
});

res.status(200).send(dataByGeostore);
return;
}

const url = `${DATA_API_URL}/${slugs.join('/')}`;
try {
const dataByUrl = await getDataFromLink({ url });

res.send(dataByUrl);
} catch (error) {
res.status(error.response?.status).send({
status: error.response?.status,
message: error?.message,
});
}
};

/**
* @param {NextApiRequest} req
* @param {NextApiResponse} res
*/
const postData = async (req, res) => {
const { query } = req;
// TODO: add more parameters to the query like, global, adm9, adm1, etc etc etc
const { slug: slugs, geostore_id, canopy_cover } = query;

if (slugs.length === 0) {
res.status(400).send();
return;
}

try {
const submitted = await createRequestByGeostoryId({
dataset: slugs[0],
geostoreId: geostore_id,
canopy: canopy_cover,
});

res.status(201).send(submitted);
} catch (error) {
res.status(error.response?.status).send({
status: error.response?.status,
message: error?.message,
});
}
};

/**
* @param {NextApiRequest} req
* @param {NextApiResponse} res
*/
export default async (req, res) => {
switch (req.method) {
case 'POST':
postData(req, res);
break;
case 'GET':
fetchDataByDatasetAndGeostore(req, res);
break;
default:
res.send(405);
}
};
38 changes: 12 additions & 26 deletions services/datamart.js
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
import axios from 'axios';
import qs from 'qs';
import { dataRequest } from 'utils/request';

@@ -22,13 +21,13 @@ import { dataRequest } from 'utils/request';
/**
* 1
* @param {Object} request - request
* @param {string} request.dataset - dataset.
* @param {string} request.geostoreId - a geostore id.
* @param {number} request.canopy - canopy filter.
* @returns {Promise<GetResponseObject | NotFoundObject>} response.
*/
export const getDataByGeostoreId = async ({ geostoreId, canopy }) => {
// TODO: receive url as a param (or at least the dataset)
const url = '/v0/land/tree-cover-loss-by-driver';
export const getDataByGeostoreId = async ({ dataset, geostoreId, canopy }) => {
const url = `/v0/land/${dataset}`;
const params = {
geostore_id: geostoreId,
canopy_cover: canopy,
@@ -58,46 +57,33 @@ export const getDataByGeostoreId = async ({ geostoreId, canopy }) => {
/**
* 2
* @param {Object} request - request
* @param {string} request.dataset - dataset.
* @param {string} request.geostoreId - a geostore id.
* @param {number} request.canopy - canopy filter.
* @returns {Promise<GetResponseObject>} response.
*/
export const createRequestByGeostoryId = async ({ geostoreId, canopy }) => {
// TODO: receive url as a param (or at least the dataset)
const url = '/v0/land/tree-cover-loss-by-driver';
export const createRequestByGeostoryId = async ({
dataset,
geostoreId,
canopy,
}) => {
const url = `/v0/land/${dataset}`;
const params = {
geostore_id: geostoreId,
canopy_cover: canopy,
};

console.log('POSTing...');
const response = await dataRequest.post(url, params);

return response;
};

const DATA_API_KEY = process.env.NEXT_PUBLIC_GFW_API_KEY;
/**
* 3
* @param {Object} request - request
* @param {string} request.url - url
* @returns {Promise<GetResponseObject>} response.
*/
export const getDataFromLink = async ({ url }) => {

console.log('> getDataFromLink: ', url)
try {
const response = await axios.create({
timeout: 30 * 1000,
headers: {
'x-api-key': DATA_API_KEY,
},
transformResponse: [(data) => JSON.parse(data)?.data],
}).get(url);
console.log('>> getDataFromLink response: ', response);
return response;
} catch (error) {
console.log('> err: ', error);
throw error;
}
export const getDataFromLink = ({ url }) => {
return dataRequest.get(url);
};