- AMQP
- Automation
- AWS
- CONFIG
- INFRA
- DELL
- DOCKER
- Monitoring
- Operating System
- OpenStack
- Rackspace
- HEROKU
- NEWS
- OpenShift
- SECURITY
- Vagrant
- VM
- Yaksha
- Xample-Tech-Stack
- http://en.wikipedia.org/wiki/Category:Cloud_infrastructure
- http://en.wikipedia.org/wiki/Category:Cloud_computing_providers
- http://en.wikipedia.org/wiki/Comparison_of_open-source_configuration_management_software
- http://docs.python-guide.org/en/latest/scenarios/admin/
- Celery, https://github.com/celery/celery
- http://docs.celeryproject.org/en/latest/getting-started/first-steps-with-celery.html#rabbitmq
- ZeroMQ, https://github.com/zeromq/pyzmq
- http://zeromq.org/intro:get-the-software
- zguide, http://zguide.zeromq.org/page:all
- guide, http://zguide.zeromq.org/py:all
- RabbitMQ, https://www.rabbitmq.com # Erlang AMQP lib
- Pika, https://github.com/pika/pika : a Pure Python RabbitMQ/AMQP 0-9-1 client library.
- Documentation: http://docs.ansible.com/ansible/git_module.html
- ansible/ansible#7030
- ansible/ansible-modules-core#55
- http://stackoverflow.com/questions/32870417/how-to-add-update-git-tag-from-local-machine-using-ansible-playbook
Makefile samples
- https://github.com/tlnagy/dotfiles/blob/master/Makefile
- http://www.jonzelner.net/statistics/make/reproducibility/2016/06/01/makefiles/
(.rb)
- https://github.com/chef/chef
- http://stackful-dev.com/cuisine-the-lightweight-chefpuppet-alternative
(.rb)
- https://docs.saltstack.com/en/latest/topics/installation/index.html
- http://bencane.com/2013/09/03/getting-started-with-saltstack-by-example-automatically-installing-nginx/
- http://jensrantil.github.io/salt-vs-ansible.html
- http://probably.co.uk/puppet-vs-chef-vs-ansible.html
- http://ryandlane.com/blog/2014/08/04/moving-away-from-puppet-saltstack-or-ansible/
- https://github.com/NARKOZ/hacker-scripts :: git clone [email protected]:NARKOZ/hacker-scripts.git
(.rb), If you mess-up the VM, blow it away with vagrant
(.py)
- https://en.wikipedia.org/wiki/Cobbler_%28software%29
- https://github.com/cobbler/cobbler
- http://aws.amazon.com
- http://en.wikipedia.org/wiki/Amazon_Web_Services
- http://en.wikipedia.org/wiki/Amazon_Elastic_Compute_Cloud
- REF : http://docs.aws.amazon.com/general/latest/gr/Welcome.html
- http://docs.aws.amazon.com/general/latest/gr/rande.html
- https://aws.amazon.com/documentation/
- CLI tools : http://docs.aws.amazon.com/general/latest/gr/GetTheTools.html
- http://stackoverflow.com/questions/18700295/standard-way-of-creating-config-file-suitable-for-python-and-java-together
- https://wiki.python.org/moin/ConfigParserExamples
- https://docs.python.org/3.5/library/configparser.html
- Luigi :: A Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
- An introduction to containers for continuous integration : https://github.com/fredhutchio/containers-ci
- Services vs Libraries: tradeoffs in dependency management.
- A Python library for interacting with many of the popular cloud service providers using a unified API - can handle files on AWS, Dropbox, GoogleDrive.
- List of supported providers : https://libcloud.readthedocs.org/en/latest/supported_providers.html
- http://www.opencompute.org :: The Open Compute Project (OCP) is reimagining hardware, making it more efficient, flexible, and scalable.
An open platform for network services across a spectrum of hardware in multivendor environments.
- Networking-Zine by @jvns.
- Everyday Hacks for Docker: https://hackernoon.com/everyday-hacks-for-docker-37e92188d9e8#.iybrog2gd
- Install docker on debian, https://docs.docker.com/engine/installation/linux/debian/
- https://docs.docker.com/v1.5/faq/
- http://stackoverflow.com/questions/16047306/how-is-docker-different-from-a-normal-virtual-machine
- https://github.com/ddboline/docker_scripts :: Scripts to use with Docker, along with example Dockerfile's
- Various Dockerfiles : https://github.com/jfrazelle/dockerfiles and a blog post: https://blog.jessfraz.com/post/docker-containers-on-the-desktop/
- https://www.digitalocean.com/community/tutorials/docker-explained-how-to-containerize-python-web-applications
- https://books.google.co.in/books?id=oAkZBQAAQBAJ&pg=PA116&lpg=PA116&dq=python+code+to+spin+up+docker+containers&source=bl&ots=dSp_ddqppk&sig=BfOcQ7hY1oBndYT3DtVbgaT8dis&hl=en&sa=X&ei=Q9EcVeroFI-0uATYvoKYBg&ved=0CEcQ6AEwBw#v=onepage&q=python%20code%20to%20spin%20up%20docker%20containers&f=true
- http://phusion.github.io/baseimage-docker/
- https://pypi.python.org/pypi/docker-compose/1.2.0rc3
- https://coreos.com/os/docs/latest/getting-started-with-docker.html
- https://docs.docker.com/installation/ubuntulinux/
- https://docs.docker.com/articles/basics/
- http://www.liquidweb.com/kb/how-to-install-docker-on-ubuntu-14-04-lts/
- dockerspawner :: Spawns JupyterHub user servers in Docker containers.
- https://www.stgraber.org/2013/12/20/lxc-1-0-blog-post-series/
- https://www.linux.com/news/event/elcna/2017/why-native-docker-orchestration-best-orchestration
- https://dev.to/derickbailey/10-myths-about-docker-that-stop-developers-cold
- http://doc.gitlab.com/omnibus/docker/
- https://github.com/sameersbn/docker-gitlab-ci
- Docs for managing data in containers: https://docs.docker.com/engine/tutorials/dockervolumes/
- WHITE PAPER : Understanding Docker Data Storage and Persistence
- http://www.computerweekly.com/feature/Docker-storage-101-How-storage-works-in-Docker
- http://stackoverflow.com/questions/18496940/how-to-deal-with-persistent-storage-e-g-databases-in-docker
- Docker - Persistence
- Dockerized Postgresql Development Environment
- http://thenewstack.io/docker-fork-talk-split-now-table/
- http://www.zdnet.com/article/red-hat-ubuntu-and-docker-container-virtualization-goes-mainstream/
- http://www.infoworld.com/article/2609888/virtualization/docker-challenges-virtualization-market-with-containers.html
- http://www.techrepublic.com/article/containers-replacements-or-alternatives-to-virtual-machines/
- http://www.zdnet.com/article/docker-an-open-source-startup-you-need-to-know-about/
- QSFT/mixcoatl#207
- https://github.com/irvingpop/enstratius-api-tools
- https://github.com/zomGreg/riak-cli-tool
- https://en.wikipedia.org/wiki/Comparison_of_network_monitoring_systems
- Blog on incron, http://www.splitbrain.org/blog/2011-01/07-watcher_a_recursive_incron_alternative
- Watcher :: A daemon that watches specified files/folders for changes and fires commands in response to those changes. It is similar to incron, however, configuration uses a simpler to read ini file instead of a plain text file. Unlike incron it can also recursively monitor directories. It's also written in Python, making it easier to hack.
- https://pypi.python.org/pypi/psutil
- https://github.com/nicolargo/glances
- https://github.com/google/grr
- https://github.com/python-diamond/Diamond
- https://github.com/Jahaja/psdash
- https://github.com/prometheus
- http://www.zenoss.com
- https://en.wikipedia.org/wiki/Ganglia_%28software%29
- https://en.wikipedia.org/wiki/Icinga
- https://en.wikipedia.org/wiki/Nagios
- CapROS : https://en.wikipedia.org/wiki/CapROS
- Google Fuchsia : https://en.wikipedia.org/wiki/Google_Fuchsia
- Nova, https://github.com/openstack/nova
- http://www.openstack.org/software/
- OpenStack drivers
- http://www.ibm.com/developerworks/cloud/library/cl-openstack-pythonapis/
- https://addons.heroku.com/
- https://devcenter.heroku.com/articles/getting-started-with-python
- https://devcenter.heroku.com/articles/procfile
- https://devcenter.heroku.com/categories/heroku-architecture
- https://id.heroku.com/login
- http://www.networkworld.com/article/2160016/cloud-computing/who-makes-up-openstack-.html
- https://training.linuxfoundation.org/sysadmin-evolution
- http://www.networkworld.com/article/2160163/cloud-computing/12-free-cloud-storage-options.html
- http://help.openshift.com
- https://blog.openshift.com/run-your-nodejs-projects-on-openshift-in-two-simple-steps/
- https://wiki.python.org/moin/FreeHosts
- credential leaks : https://github.com/ChALkeR/notes/blob/master/Do-not-underestimate-credentials-leaks.md
- https://en.wikipedia.org/wiki/Operations_security
- Use OTR for chatting, GnuPG for encrypted & verified data sharing, with EnigMail for emails and TrueCrypt for storage.
- Encrypt your Flash Drives with Linux: https://www.linux.com/learn/easily-encrypt-your-flash-drives-linux
- Discover subdomains with automated tools:
- Subbrute
- Virustotal.com
- Dnsdumster.com
- Dnsdb.org
- Censys.io
- CSP headers
Each line should start with $ipt (which is your variable that points to the iptables binary with complete path). This shows your second line starting with "ESTABLISHED" & third line with "$WAN_IFACE".
Did you check your linefeeds and command formatting?
Alternately flush your tables and load each command manually to see where/if you have script errors! Just cut/copy/paste each line into your tables, then display the full recipe with "iptables_save" and/or pipe via STOUT to a text file to compare or import at startup. Depending on your distro, iptables startup config and version, your templates might conflict. Many stock firewalls, VPN solutions and Virtualized servers use bridging, TUN or null devices with DNAT/SNAT and or EBTABLES.
Just pulling partial iptables out of your scripts tool bin for use, while replacing the complete startup UFW config firewall, might be a quick solution, however security threats today demand that you use a complete solution or build upon modern tools, like SHOREWALL: http://shorewall.net/bridge-Shorewall-perl.html
Other appliance based SOHO firewalls include: SMOOTHWALL, IPCOP.
- Firewalld, OpenWall, IPWire, MonoWall, & Untangle are compared here: https://en.m.wikipedia.org/wiki/Comparison_of_firewalls
- Block and limit known attacks: http://www.cyberciti.biz/tips/linux-iptables-10-how-to-block-common-attack.html
Also don't forget to check your kernel values! In order to support iptables in bridging mode (which you are not doing but might consider), your Linux kernel needs to be compiled with CONFIG_BRIDGE_NETFILTER=1, and your /etc/sysctl.conf file either needs to not contain any entries for the following settings or have them set to “1”:
net.bridge.bridge-nf-call-arptables=0 net.bridge.bridge-nf-call-ip6tables=0 net.bridge.bridge-nf-call-iptables=0
Traditionally, Ubuntu and other Debian derivatives store network interface configurations in the file /etc/network/interfaces. However, GNOME's Network Manager system automatically configures any interface not explicitly described in that file.
In theory, this should mean that if you specify interface and bridge configurations in /etc/network/interfaces, you shouldn't have to worry
about Network Manager overriding or otherwise conflicting with those settings. But in practice, most Admins agree, you're better off
disabling Network Manager altogether in the System→Preferences→Startup Applications applet, if you want to set up a bridged iptable configuration in /etc/network/interfaces.
To completely disable Network Manager, you also need to open the System→Preferences→Network Connections control panel and delete all
connection profiles under the Wired tab. Even if Network Manager is disabled as a startup service, Ubuntu will read network configuration
information set by this control panel, resulting in strange interactions with /etc/network/interfaces.
Even after disabling the Network Manager service, setting up /etc/network/interfaces and stopping and restarting /etc/init.d/networking,
ethernet devices can show up in the routing table with the same IP address as the bridge (which is why we asked you what your route was showing). You must kill all processes and restart the Daemon.
Since most of us really need something "bigger" than say as SOHO router (even something running a ddwrt /openwrt solution), I would suggest adding a nice appliance instead?
This is a pretty tight solution: REDWALL, http://redwall.sourceforge.net/features.html
- https://www.owasp.org/index.php/Main_Page, is a good place to start for Web, aimed more towards the Development side (applications).
- Linux server hardening, http://www.tecmint.com/linux-server-hardening-security-tips/ Keep in mind those tips are for PRODUCTION SERVERS rather than Development machines or Workstations. For example, it is best practices NOT to have GUI desktops on most Production Linux Servers (there are some exceptions), but for Workstations, most Developers would prefer to have a GUI development environment.
- setting up a Debian VM step by step instructions: https://jtreminio.com/2012/07/setting-up-a-debian-vm-step-by-step/
- gitian-builder :: Build packages in a secure deterministic fashion inside a VM. https://gitian.org/
- Papertrail, CopperEgg, Skeddly, Monit, Jenkins.
- https://github.com/SUSE-Cloud/automation Dotbot
- https://github.com/webpro/awesome-dotfiles
- https://github.com/cypher/dotfiles
- https://github.com/TheLocehiliosan/yadm
- http://dotfiles.github.io/
- Blog post on managing dotfiles and dotbot with the dotfiles template.
- Shell script to initialize your dotfiles with
dotbot: https://github.com/Vaelatern/init-dotfiles - https://github.com/szaghi/dotfiles
- https://bitbucket.org/patrickdoane/dotfiles
- https://github.com/drewfrank/dotfiles homeshick
- homeshick : git dotfiles synchronizer written in bash.
- https://github.com/andsens/rc-files : homesick compatible bashrc and zshrc files.
- @holman talks about keeping dotfiles lean. Learn how they do it: rtomayko, holman, mathiasbynens, .. zsh
- Reconsider if zsh is more useful than bash? on all machines?
- https://github.com/jbernard/dotfiles :: Dotfile management made easy. http://pypi.python.org/pypi/dotfiles/
- https://github.com/alebcay/awesome-shell AWS
- https://github.com/colinbjohnson/aws-missing-tools :: tools for managing AWS resources including EC2, EBS, RDS, IAM, CloudFormation and Route53.
- https://github.com/colinbjohnson/snippets
- https://github.com/colinbjohnson/awstraining BASH
- basher :: A package manager for shell scripts.
- Bash8 :: A pep8 equivalent for bash scripts.
- HuffShell :: A gem for suggesting and optimizing your shell aliases, programtically.
- https://github.com/tsileo/dirtools :: Exclude/ignore files in a directory (using .gitignore like syntax), compute hash, search projects for an entire directory tree, gzip compression and track changes in a directory over time.
- https://github.com/tsileo/incremental-backups-tools :: Storage agnostic incremental backups tools, building blocks for creating incremental backups utilities.
- https://github.com/lra/mackup :: Keep your application settings in sync (OS X/Linux).
- https://pypi.python.org/pypi/backalaika/0.2.1
- https://www.zufallsheld.de/2014/01/25/python-backup-script-revisited/
- https://github.com/aliceh75/dufl
- https://github.com/ContinuumIO/docker-images : Repository of Docker images created by Continuum Analytics.
- See the Docker site: https://registry.hub.docker.com/repos/continuumio/
- https://github.com/massie/dockerfiles
- Safely store secrets in Git/Mercurial : https://github.com/StackExchange/blackbox
- Uber tech stack: https://eng.uber.com/tech-stack-part-one/