Custom Kernel Images loading on Enterprise Gateway

[j-raghavan]

Hello Team,

I have built a custom Kernel Image for EG and pushed it to docker.io My objective is to launch Enterprise Gateway with my custom kernel.

When I try to install it via Helm, I am seeing the issue that the Kernel spec volume mount not found.

root@ip-10-60-1-119:/home/ubuntu# helm upgrade --install --create-namespace eg --namespace eg jeg Release "eg" does not exist. Installing it now. Error: Deployment.apps "enterprise-gateway" is invalid: [spec.template.spec.containers[1].volumeMounts[0].name: Not found: "image-kernelspecs", spec.template.spec.initContainers[0].volumeMounts[0].name: Not found: "image-kernelspecs"] root@ip-10-60-1-119:/home/ubuntu#

Here is a snippet from my values.yaml file that I have used to launch the EG.

**kernel:
clusterRole: kernel-controller
shareGatewayNamespace: false
launchTimeout: 60
cullIdleTimeout: 3600
whitelist:
- r_kubernetes
- python_kubernetes
- python_tf_kubernetes
- python_tf_gpu_kubernetes
- scala_kubernetes
- spark_r_kubernetes
- spark_python_kubernetes
- spark_scala_kubernetes
-
defaultKernelName: python_kubernetes

kernelspecs:
image: docker.io/mrjayasimha/jupyt3r_eg_kernel:0.1.0
imagePullPolicy: Always**

Here is my deployment.yaml

**apiVersion: apps/v1
kind: Deployment
metadata:
name: enterprise-gateway
namespace: {{ .Release.Namespace }}
labels:
gateway-selector: enterprise-gateway
app: enterprise-gateway
component: enterprise-gateway
chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "" }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
spec:
replicas: {{ .Values.replicas }}
selector:
matchLabels:
gateway-selector: enterprise-gateway
template:
metadata:
labels:
gateway-selector: enterprise-gateway
app: enterprise-gateway
component: enterprise-gateway
spec:
serviceAccountName: enterprise-gateway-sa
{{- if .Values.kernelspecs.image }}
initContainers:
- name: kernelspecs
image: {{ .Values.kernelspecs.image }}
imagePullPolicy: {{ .Values.kernelspecs.imagePullPolicy }}
args: ["cp", "-r", "/kernels", "/usr/local/share/jupyter"]
volumeMounts:
- name: image-kernelspecs
mountPath: "/usr/local/share/jupyter/kernels"**_

I know documentation is scarce hoping someone would advise what am I missing here.

Best Regards,
Jayasimha

[kevin-bates]

Hi @j-raghavan. Are you able to launch any of the out-of-the-box kernel images that come configured with EG? I would focus on getting python_kubernetes working in a similar configuration to what you are targeting if you haven't done so already.

It looks like the image you created (docker.io/mrjayasimha/jupyt3r_eg_kernel:0.1.0) is intended to be an init-container containing all of your kernelspec references by virtue of it being referenced in the values yaml as the kernelspecs.image. This kind of image merely houses kernel specifications - similar to what you find in the EG container under /usr/local/share/jupyter/kernels and is not considered a "kernel image".

Two things must occur for a custom kernel image to be used.

1. The kernel image must be built according to the docs, with the primary pieces being the presence of the bootstrap_kernel.sh script and a kernel launcher script.
2. The kernelspec file (kernel.json) must be properly configured and made available to EG. This is accomplished either by extending the EG image to include your additional kernelspec set of files that reference your kernel image. Or by using one of the mounting approaches (nfs, init-container, etc.).

For starters, besides making sure the basics are working in your configuration, you might want to create an appropriate init-container where the kernel specifications are located in /kernels and are then copied (recursively) to /usr/local/share/jupyter/kernels in the EG container.

[j-raghavan]

Hi Kevin,

Thank you very much for the Advice. I followed it and was able to get my custom kernel loaded on my EG.
For the benefit of others, what I had to do was -

1. Create a DockerFile that builds just the kernelspec. For this you would need to download the Kernel spec file specific to the kernel (in my case it was K8) and built the kernel spec docker image.
2. Used Helm to install the EG with the custom kernelspec that I created in 1.

Here is the Docker registry page

https://hub.docker.com/u/mrjayasimha

Thanks Again!

Best regards,
Jayasimha