-
-
Notifications
You must be signed in to change notification settings - Fork 413
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conpot new features #1 #375
Conversation
* Some basic tests * Improved Virtual File System
Pull Request Test Coverage Report for Build 1042
💛 - Coveralls |
if self.cls.__name__ not in 'Proxy': | ||
core_interface.protocols[self.cls] = self.wrapped | ||
|
||
def __getattr__(self, name): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorcery using magic methods.
Implementing getattr overrides Python's default mechanism for member access. So basically every attribute of a class that is decorated by @conpot_protocol
decorator can be accessed here. This can be very powerful in case we want to emulate a system-wide phenomenon. Like for example we want to emulate a system restart (kamstrup management protocol ;-) we can set a counter here and freeze access to all protocols.
Some other uses include timing the last attack. This can be done by tracking the handle method for every protocol. Again can be easily done from here, without even touching the protocol implementation :-)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
that sounds really nice to me -
@glaslos would you mind to think this through for a second? :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
def __repr__(self): | ||
return self.cls.__repr__(self.wrapped) | ||
|
||
__doc__ = property(lambda self: self.cls.__doc__) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well the downside of this approach is - every object that we create in bin/conpot
is an instance this Wrapper
class and no longer the instance of the actual protocol class. As you can see, I am monkey-patching
the important stuff. So too many changes here and there could lead to very unexpected results.
However, unplugging a protocol from the internal interface is as easy as removing the @conpot_protocol
decorator. Everything would still run perfect.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For now I see the benefits and I am really curious how this will affect subsequent development (in terms of making things more flexible and easier to tie together).
A last question on this matter: How do you (personally) feel about debugging? Unexpected results is one thing - but I am a little unsure if this will have negative impact on how straight forward debugging issues will be in future :) So if somebody has any strong opinions on this specific matter, I'm open for input :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My objective for internal interface has always been to let protocols interact more deeply with each other. Not just for debugging. So for example if users want an option to change databus values while Conpot is still running, they can do so - if we extend the interface ;-)
IMO as long we maintain a plug/unplug mechanism, there is little to worry about. We can always go for - 'X' protocols with interface and 'Y' protocols without it. Conpot would still run all protocols like nothing has changed.
|-- common | ||
|-- telnet | ||
|-- http | ||
|-- snmp | ||
`-- ftp etc. | ||
""" | ||
def __init__(self): | ||
self.data_fs = osfs.OSFS(root_path='') | ||
self.data_fs = osfs.OSFS(root_path='') # Specify the place where you would place the uploads |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
pyfilesystem2
is pretty cool. You can specify something like ftp://will:[email protected]/private
to store the uploads directly to a FTP server.
You can even do this - zip://projects.zip
or tar://projects.tar
to store directly to a zip/tar file. I haven't tested these URLs though.
Question: What should we keep the deafult storage URL here? Since we are creating an instance of this VirtualFS class in conpot.core.__init__.py
, I was not sure about the default file, directory - could use some advice here ;-)
@creolis
cc: @glaslos
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You mean where the default directory for the VFS will reside?
$cwd/vfs seems appropriate, ... or do I miss an obvious tripwire?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We would need to push this default directory to conpot repo. Would a conpot_vfs.tar
file here be okay with you?
logger = logging.getLogger(__name__) | ||
|
||
|
||
class ProxyDecoder(abc.ABC): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ABCs for pluggable proxy decoders
conpot/core/file_io.py
Outdated
- Use AbstractFS.home_fs object to access the chroot jail FS | ||
- Use AbstractFS.root_fs object to access the global '/' FS | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some really good stuff I thought of when I was beginning to experiment with FTP. Use AbstractFS.root_fs
to access '/'
of the vfs and use AbstractFS.home_fs
to access a protocol specific chroot jail FS. AbstractFS.home_fs
is basically /' + 'protocol_name'
directory. But if a protocol tries to access ../
from home_fs
, an implicit Illegal back reference exception is raised.
AbstractFS also wraps around os.* command. So you can do AbstractFS.mkdir(path)
etc.
Made a mistake here: Use AbstractFS.home_fs object attribute to access the chroot jail FS.
raise NotImplementedError | ||
|
||
def chmod(self, path, mode): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Simulating chmod
in vfs : Shoud we allow this? If not what should we return?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, we should definitely allow this. Some protocols (like FTP) will make use of it and it might be very strange if you're able to upload, but not able to change permissions (unless you're all of the sudden not the owner of the file anymore)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But wouldn't it expose us to privledge escalation exploits? Since we're accepting arbitary uploads from potential hackers ;-) I was thinking about maybe faking it somehow.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, I see. I was still thinking of the final VFS approach as a "really" virtual filesystem where all content it stored in virtual files and permissions / other metadata is stored held within memory (or stored permanently in a descriptor file) in order to simulate file access. Of course we should not render ourselves prone to exploitation - but we have to provide the current set of typical FTP commands - and chmod is usually allowed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well, pyfilesystem would allow us to create FS in memory. It is called memoryFS
in their terms.
Actually, storing FS in memory could be better than storing FS on disk (faster access etc.). However FS can be bulky (given the number of protocols), do you think is is okay have that on RAM?
There would be an additional overhead of flushing the filesystem when conpot crashes.
The current implementation creates the protocol_fs in /tmp
. So we don't have to worry about anything.
I'll have a look into how this can be faked. I'll let you know about any findings :-)
* Guardian AST tests * Modbus function 43 * Several 501 and 404 tests for HTTP server * Proxy decoder tests
@@ -12,6 +10,13 @@ omit = | |||
conpot/core/loggers/stix_transform.py | |||
conpot/core/loggers/taxii_log.py | |||
conpot/utils/mac_addr.py | |||
# New features.. Just to avoid coverage drop - before we actually write some tests |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you're starting to take drops in coverage personally, do you? ;)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hehe. No. Not really 😉
I merged, even though there is one open question, but this is more a thing to debate on and maybe add some notes to the docs. In general, since there is the decorator now, I think we should add a general description and "DOs and DONTs" for future developers to the docs, hinting them about possible tripwires. All in all I'm more then happy with the PR <3 |
Thanks for the merge :-) |
Conpot new features mushorg#1
Second GSoC PR 🎉 This PR is more like a status update dealing with:
This PR does contain some 'controversial' code. I'll mark those so that we can have a discussion ;-)
NOTE:
Based on my past experience with tests, I am experimenting with something called TDD. So I am planning to write FTP tests first - even before fully implementing it.
Also, I usually squash my commits to keep the repo clean. But maybe I should commit more often?