Update instructions for AWS credentials in ArXiv download and extract tutorial

tutorials/text/download-and-extract/download_extract_tutorial.ipynb

@@ -39,11 +39,33 @@
     "!apt install -y wget"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "abffed09",
+   "metadata": {},
+   "source": [
+    "If running the ArXiv Download and Extract section of this notebook, you must set your AWS credentials here:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "08e15d88",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%env AWS_ACCESS_KEY_ID=\n",
+    "%env AWS_SECRET_ACCESS_KEY=\n",
+    "%env AWS_SESSION_TOKEN="
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "c6c131c1",
    "metadata": {},
    "source": [
+    "To avoid credentials-related errors, you must specify the above environment variables **before** starting the Ray client (see below). Do not include quotes when setting your credentials (e.g., it should be `%env AWS_ACCESS_KEY_ID=XYZ` and not `%env AWS_ACCESS_KEY_ID=\"XYZ\"`).\n",
+    "\n",
     "To run a pipeline in NeMo Curator, we must start a Ray cluster. This can be done manually (see the [Ray documentation](https://docs.ray.io/en/latest/ray-core/starting-ray.html)) or with Curator's `RayClient`:"
    ]
   },
@@ -398,34 +420,8 @@
     "More information about downloading ArXiv data with NeMo Curator can be found in the [documentation](https://docs.nvidia.com/nemo/curator/latest/curate-text/load-data/arxiv.html).\n",
     "\n",
     "### Prerequisites\n",
-    "Before proceeding, we need to set **AWS credentials** in the environment as the data are stored in AWS S3 [requester pays buckets](https://docs.aws.amazon.com/AmazonS3/latest/userguide/RequesterPaysBuckets.html) (pricing details could be found [here](https://aws.amazon.com/s3/pricing/))."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "ef101752-aa99-470a-b587-4548b83c1625",
-   "metadata": {},
-   "source": [
-    "#### Set your AWS Credentials"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "193812d1-69b0-46d7-a1f4-525e52573671",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "%env AWS_ACCESS_KEY_ID=\n",
-    "%env AWS_SECRET_ACCESS_KEY=\n",
-    "%env AWS_SESSION_TOKEN="
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "988105cf-5b4e-4288-be1d-4c4219c98bf9",
-   "metadata": {},
-   "source": [
+    "Before proceeding, we need to set **AWS credentials** in the environment as the data are stored in AWS S3 [requester pays buckets](https://docs.aws.amazon.com/AmazonS3/latest/userguide/RequesterPaysBuckets.html) (pricing details could be found [here](https://aws.amazon.com/s3/pricing/)). Credentials should be set at the top of this notebook, before starting a Ray client object.\n",
+    "\n",
     "#### Ensure the ArXiv S3 bucket accessibility"
    ]
   },
@@ -438,7 +434,19 @@
    },
    "outputs": [],
    "source": [
-    "!s5cmd --request-payer=requester ls s3://arxiv/src/ | head | grep '.tar'"
+    "import subprocess\n",
+    "\n",
+    "cmd = \"s5cmd --request-payer=requester ls s3://arxiv/src/ | head | grep '.tar'\"\n",
+    "\n",
+    "# shell=True needed because we use pipes\n",
+    "result = subprocess.run(cmd, shell=True, capture_output=True, text=True, check=True)  # noqa: S602\n",
+    "\n",
+    "# stdout has the command output\n",
+    "print(result.stdout)\n",
+    "\n",
+    "# stderr has errors\n",
+    "if result.stderr:\n",
+    "    print(\"Errors:\", result.stderr)"
    ]
   },
   {