Skip to content

simplify download server #307

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
rphillips wants to merge 1 commit into from
Closed

simplify download server #307

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 10 additions & 31 deletions manifests/07-downloads-deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,37 +59,22 @@ spec:
- '-c'
- |
cat <<EOF >>/tmp/serve.py
import BaseHTTPServer, os, re, signal, SimpleHTTPServer, socket, sys, tarfile, tempfile, threading, time, zipfile
from BaseHTTPServer import HTTPServer, BaseHTTPRequestHandler
from SocketServer import ThreadingMixIn
import SimpleHTTPServer, os, re, signal, sys, tarfile, tempfile, zipfile

class ThreadedHTTPServer(ThreadingMixIn, HTTPServer):
Copy link
Member

@wking wking Oct 4, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added the bulk of this script in #177. As discussed there, it was largely a copy-paste from ci-operator. @smarterclayton added that code on 2018-06-25 in openshift/ci-operator#35. I'm not familiar enough with its source to know what it's all about, but he links to https://stackoverflow.com/questions/46210672/ below and that says:

No, I don't want a hack such as ThreadingMixIn that merely collects up the response and returns it as a unit.

So I expect he was aware of this approach and has consciously decided against it. But maybe there have been stdlib improvements in 2.7 since that SO post? I haven't hunted deeply enough to know.

Copy link
Author

@rphillips rphillips Oct 8, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand the original motivation... The main thread accepts the client and uses a thread to handle the I/O of the client. There is not a streaming interface with the current code.

Copy link
Contributor

@benjaminapetersen benjaminapetersen Oct 8, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am hesitant to do a ton with this w/o test coverage.
We will have an e2e test coming soon with #306, probably, which will indirectly help ensure this component doesn't fall over.

pass

signal.signal(signal.SIGTERM, lambda signum, frame: sys.exit(0))

# Launch multiple listeners as threads
class Thread(threading.Thread):
def __init__(self, i, socket):
threading.Thread.__init__(self)
self.i = i
self.socket = socket
self.daemon = True
self.start()

def run(self):
httpd = BaseHTTPServer.HTTPServer(addr, SimpleHTTPServer.SimpleHTTPRequestHandler, False)

# Prevent the HTTP server from re-binding every handler.
# https://stackoverflow.com/questions/46210672/
httpd.socket = self.socket
httpd.server_bind = self.server_close = lambda self: None

httpd.serve_forever()


temp_dir = tempfile.mkdtemp()
print('serving from {}'.format(temp_dir))
os.chdir(temp_dir)
for arch in ['amd64']:
os.mkdir(arch)
for operating_system in ['linux', 'mac', 'windows']:
os.mkdir(os.path.join(arch, operating_system))

for arch, operating_system, path in [
('amd64', 'linux', '/usr/bin/oc'),
('amd64', 'mac', '/usr/share/openshift/mac/oc'),
Expand All @@ -104,16 +89,10 @@ spec:
tar.add(path, basename)
with zipfile.ZipFile('{}.zip'.format(archive_path_root), 'w') as zip:
zip.write(path, basename)

# Create socket

addr = ('', 8080)
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind(addr)
sock.listen(5)

[Thread(i, socket=sock) for i in range(100)]
time.sleep(9e9)
server = ThreadedHTTPServer(addr, SimpleHTTPServer.SimpleHTTPRequestHandler)
server.serve_forever()
EOF
exec python2 /tmp/serve.py # the cli image only has Python 2.7
terminationGracePeriodSeconds: 1