Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Pyobjects (formerly NaCl) to Salt #10517

Merged
merged 5 commits into from
Feb 18, 2014
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
158 changes: 158 additions & 0 deletions salt/renderers/pyobjects.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@
# -*- coding: utf-8 -*-
'''
:maintainer: Evan Borgstrom <[email protected]>

Python renderer that includes a Pythonic Object based interface

Let's take a look at how you use pyobjects in a state file. Here's a quick
example that ensures the ``/tmp`` directory is in the correct state.

.. code-block:: python
:linenos:
#!pyobjects

File.managed("/tmp", user='root', group='root', mode='1777')

Nice and Pythonic!

By using the "shebang" syntax to switch to the pyobjects renderer we can now
write our state data using an object based interface that should feel at home
to python developers. You can import any module and do anything that you'd
like (with caution, importing sqlalchemy, django or other large frameworks has
not been tested yet). Using the pyobjects renderer is exactly the same as
using the built-in Python renderer with the exception that pyobjects takes
care of creating an object for each of the available states on the minion.
Each state is represented by an object that is the capitalized version of it's
name (ie. ``File``, ``Service``, ``User``, etc), and these objects expose all
of their available state functions (ie. ``File.managed``, ``Service.running``,
etc).

Context Managers and requisites
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
How about something a little more complex. Here we're going to get into the
core of what makes pyobjects the best way to write states.

.. code-block:: python
:linenos:
#!pyobjects

with Pkg.installed("nginx"):
Service.running("nginx", enable=True)

with Service("nginx", "watch_in"):
File.managed("/etc/nginx/conf.d/mysite.conf",
owner='root', group='root', mode='0444',
source='salt://nginx/mysite.conf')


The objects that are returned from each of the magic method calls are setup to
be used a Python context managers (``with``) and when you use them as such all
declarations made within the scope will **automatically** use the enclosing
state as a requisite!

The above could have also been written use direct requisite statements as.

.. code-block:: python
:linenos:
#!pyobjects

Pkg.installed("nginx")
Service.running("nginx", enable=True, require=Pkg("nginx"))
File.managed("/etc/nginx/conf.d/mysite.conf",
owner='root', group='root', mode='0444',
source='salt://nginx/mysite.conf',
watch_in=Service("nginx"))

You can use the direct requisite statement for referencing states that are
generated outside of the current file.

.. code-block:: python
:linenos:
#!pyobjects

# some-other-package is defined in some other state file
Pkg.installed("nginx", require=Pkg("some-other-package"))

The last thing that direct requisites provide is the ability to select which
of the SaltStack requisites you want to use (require, require_in, watch,
watch_in, use & use_in) when using the requisite as a context manager.

.. code-block:: python
:linenos:
#!pyobjects

with Service("my-service", "watch_in"):
...

The above example would cause all declarations inside the scope of the context
manager to automatically have their ``watch_in`` set to
``Service("my-service")``.

Including and Extending
^^^^^^^^^^^^^^^^^^^^^^^

To include other states use the Include() function. It takes one name per
state to include.

To extend another state use the Extend() function on the name when creating
a state.

.. code-block:: python
:linenos:
#!pyobjects

Include('http', 'ssh')

Service.running(Extend('apache'),
watch=[{'file': '/etc/httpd/extra/httpd-vhosts.conf'}])
'''

import logging

from salt.loader import states
from salt.utils.pyobjects import StateFactory, StateRegistry

log = logging.getLogger(__name__)


def render(template, saltenv='base', sls='',
tmplpath=None, rendered_sls=None,
_states=None, **kwargs):

_registry = StateRegistry()
if _states is None:
_states = states(__opts__, __salt__)

# build our list of states and functions
_st_funcs = {}
for func in _states:
(mod, func) = func.split(".")
if mod not in _st_funcs:
_st_funcs[mod] = []
_st_funcs[mod].append(func)

# create our StateFactory objects
for mod in _st_funcs:
_st_funcs[mod].sort()
mod_upper = mod.capitalize()
mod_cmd = "%s = StateFactory('%s', registry=_registry, valid_funcs=['%s'])" % (
mod_upper, mod,
"','".join(_st_funcs[mod])
)
exec(mod_cmd)

# add our Include and Extend functions
Include = _registry.include
Extend = _registry.make_extend

# for convenience
try:
pillar = __pillar__
grains = __grains__
salt = __salt__
except NameError:
pass

exec(template.read())

return _registry.salt_data()
228 changes: 228 additions & 0 deletions salt/utils/pyobjects.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,228 @@
# -*- coding: utf-8 -*-
'''
:maintainer: Evan Borgstrom <[email protected]>

Pythonic object interface to creating state data, see the pyobjects renderer
for more documentation.
'''
from salt.utils.odict import OrderedDict

REQUISITES = ('require', 'watch', 'use', 'require_in', 'watch_in', 'use_in')


class StateException(Exception):
pass


class DuplicateState(StateException):
pass


class InvalidFunction(StateException):
pass


class StateRegistry(object):
"""
The StateRegistry holds all of the states that have been created.
"""
def __init__(self):
self.empty()

def empty(self):
self.states = OrderedDict()
self.requisites = []
self.includes = []
self.extends = OrderedDict()

def include(self, *args):
self.includes += args

def salt_data(self):
states = OrderedDict([
(id_, state())
for id_, state in self.states.iteritems()
])

if self.includes:
states['include'] = self.includes

if self.extends:
states['extend'] = OrderedDict([
(id_, state())
for id_, state in self.extends.iteritems()
])

self.empty()

return states

def add(self, id_, state, extend=False):
if extend:
attr = self.extends
else:
attr = self.states

if id_ in attr:
raise DuplicateState("A state with id '%s' already exists" % id_)

# if we have requisites in our stack then add them to the state
if len(self.requisites) > 0:
for req in self.requisites:
if req.requisite not in state.kwargs:
state.kwargs[req.requisite] = []
state.kwargs[req.requisite].append(req())

attr[id_] = state

def extend(self, id_, state):
self.add(id_, state, extend=True)

def make_extend(self, name):
return StateExtend(name)

def push_requisite(self, requisite):
self.requisites.append(requisite)

def pop_requisite(self):
del self.requisites[-1]


class StateExtend(object):
def __init__(self, name):
self.name = name


class StateRequisite(object):
def __init__(self, requisite, module, id_, registry):
self.requisite = requisite
self.module = module
self.id_ = id_
self.registry = registry

def __call__(self):
return {self.module: self.id_}

def __enter__(self):
self.registry.push_requisite(self)

def __exit__(self, type, value, traceback):
self.registry.pop_requisite()


class StateFactory(object):
"""
The StateFactory is used to generate new States through a natural syntax

It is used by initializing it with the name of the salt module::

File = StateFactory("file")

Any attribute accessed on the instance returned by StateFactory is a lambda
that is a short cut for generating State objects::

File.managed('/path/', owner='root', group='root')

The kwargs are passed through to the State object
"""
def __init__(self, module, registry, valid_funcs=[]):
self.module = module
self.valid_funcs = valid_funcs
self.registry = registry

def __getattr__(self, func):
if len(self.valid_funcs) > 0 and func not in self.valid_funcs:
raise InvalidFunction("The function '%s' does not exist in the "
"StateFactory for '%s'" % (func, self.module))

def make_state(id_, **kwargs):
return State(
id_,
self.module,
func,
registry=self.registry,
**kwargs
)
return make_state

def __call__(self, id_, requisite='require'):
"""
When an object is called it is being used as a requisite
"""
# return the correct data structure for the requisite
return StateRequisite(requisite, self.module, id_,
registry=self.registry)


class State(object):
"""
This represents a single item in the state tree

The id_ is the id of the state, the func is the full name of the salt
state (ie. file.managed). All the keyword args you pass in become the
properties of your state.

The registry is where the state should be stored. It is optional and will
use the default registry if not specified.
"""

def __init__(self, id_, module, func, registry, **kwargs):
self.id_ = id_
self.module = module
self.func = func
self.kwargs = kwargs
self.registry = registry

if isinstance(self.id_, StateExtend):
self.registry.extend(self.id_.name, self)
self.id_ = self.id_.name
else:
self.registry.add(self.id_, self)

self.requisite = StateRequisite('require', self.module, self.id_,
registry=self.registry)

@property
def attrs(self):
kwargs = self.kwargs

# handle our requisites
for attr in REQUISITES:
if attr in kwargs:
# our requisites should all be lists, but when you only have a
# single item it's more convenient to provide it without
# wrapping it in a list. transform them into a list
if not isinstance(kwargs[attr], list):
kwargs[attr] = [kwargs[attr]]

# rebuild the requisite list transforming any of the actual
# StateRequisite objects into their representative dict
kwargs[attr] = [
req() if isinstance(req, StateRequisite) else req
for req in kwargs[attr]
]

# build our attrs from kwargs. we sort the kwargs by key so that we
# have consistent ordering for tests
return [
{k: kwargs[k]}
for k in sorted(kwargs.iterkeys())
]

@property
def full_func(self):
return "%s.%s" % (self.module, self.func)

def __str__(self):
return "%s = %s:%s" % (self.id_, self.full_func, self.attrs)

def __call__(self):
return {
self.full_func: self.attrs
}

def __enter__(self):
self.registry.push_requisite(self.requisite)

def __exit__(self, type, value, traceback):
self.registry.pop_requisite()
Loading