Skare 0.15

[taldcroft]

• Ska.engarchive 0.21 to 0.22.1: Allow remote Windows access to archive (M. Baski)
• Chandra.taco 0.1 to 0.2: Allow for reproducible ACIS FP predictions
• Xija 0.2.7 to 0.3.2: Allow for different pitch bins in SolarHeat model
• Chandra.Time 1.14 to 1.15.1: Fix a bug in GRETA time conversion
• Matplotlib 1.2 to 1.2.1: bug fix release
• Pyger 0.2.7 to 0.3.0: Add Xija MinusZ model
• Astropy 0.2.1: http://astropy.org

[mbaski]

@taldcroft - I'm having issues installing this new version of xija on Windows, due to the remote access code I added in engarchive. Xija init (which is executed in its setup.py: "from xija.version import version") imports engarchive.fetch which then automatically tries to establish a connection to the remote Ska on GRETA (querying the user, which disrupts the installation process). Perhaps the remote access needs to be performed differently (not during import, though that would necessitate major changes to fetch), or can you think of other options? Perhaps we could disable remote access as part of setup.py(?)

[taldcroft]

I learned a dirty trick from the astropy crowd. In setup.py you can set a super-global via __builtin__ which can then be used in __init__.py (or any other code that gets run from there):

https://github.com/astropy/astropy/blob/master/setup.py#L28

So you basically set a flag that says you are running setup.py and don't do the remote stuff. E.g.:

https://github.com/astropy/astropy/blob/master/astropy/nddata/__init__.py

[taldcroft]

OK, the __builtin__ trick was a nice diversion, but it turned out that there is a clean simple way to avoid the problem, I believe. Please try this : https://github.com/sot/xija/pull/18/files.

[taldcroft]

@mbaski - Following on from the comment in sot/cheta#50 about testing, there are two more modules with self tests that you can now run from the python shell on installed modules:

import Chandra.Time
Chandra.Time.test()

import xija
xija.test()

I noticed that on i686 on GRETA there is one Chandra.Time test that fails:

E       AssertionError: 441763266.184 != 441763266.18399996

This is just a floating point representation issue. If it passes for you on Windows then I'm inclined to let this slide for now and I can fix up the test later.

This version of Skare 0.15 (tag 0.15-rc2) is installed on GRETA and the source tar balls should all be there.

[mbaski]

The Chandra.Time test ran fine on Windows (though I had to run it the way I normally do, through the setup.py utility due to stdin issues) - just the one difference in date representation that always occurs:

AssertionError: '01/01/99 12:13:14' != 'Fri Jan 1 12:13:14 1999'

By the way, the test_secs test with the 441763266.18399996 check passed.

[mbaski]

The xija test had issues on Windows, but the thermal models called from MCC seem to be working fine, so I'm not planning to investigate..

================================================= test session starts =================================================
platform win32 -- Python 2.7.2 -- pytest-2.2.4
collected 5 items

test_models.py s...F

====================================================== FAILURES =======================================================
____________________________________________ test_multi_solar_heat_values _____________________________________________

def test_multi_solar_heat_values():
    P_pitches = [45, 180]
    P_vals = [1.0, 1.0]
    P_pitches2 = [45, 65, 90, 140, 180]
    P_vals2 = [1.0, 1.0, 0.0, 1.0, 1.0]

    model = ThermalModel('test', start='2011:001', stop='2011:005')
    tephin = model.add(Node, 'tephin')
    tcylaft6 = model.add(Node, 'tcylaft6')
    pitch = model.add(Pitch)
    eclipse = model.add(Eclipse)

    model.add(HeatSink, tephin, T=0.0, tau=200.0)
    model.add(HeatSink, tcylaft6, T=0.0, tau=200.0)

    model.add(SolarHeat, tephin, pitch, eclipse, P_pitches, P_vals)
    model.add(SolarHeat, tcylaft6, pitch, eclipse, P_pitches2, P_vals2)

    tephin.set_data(30.0)
    tcylaft6.set_data(30.0)
    pitch.set_data(90.0)
    eclipse.set_data(False)

'> model.make()

test_models.py:145:

---

self = <xija.model.XijaModel object at 0x000000000C89CA20>

def make(self):
    """Call self.make_mvals and self.make_tmal to prepare for model evaluation
        once all model components have been added."""

'> self.make_mvals()

..\model.py:409:

---

self = <xija.model.XijaModel object at 0x000000000C89CA20>

def make_mvals(self):
    """Initialize the global mvals (model values) array.  This is an
        N (rows) x n_times (cols) array that contains all data needed
        to compute the model prediction.  All rows are initialized to
        relevant data values (e.g. node temps, time-dependent power,
        external temperatures, etc).  In the model calculation some
        rows will be overwritten with predictions.
        """
    # Select components with data values, and from those select ones that
    # get predicted and those that do not get predicted
    comps = [x for x in self.comps if x.n_mvals]
    preds = [x for x in comps if x.predict]
    unpreds = [x for x in comps if not x.predict]

    # Register the location of component mvals in global mvals
    i = 0
    for comp in preds + unpreds:
        comp.mvals_i = i
        i += comp.n_mvals

    # Stack the input dvals.  This *copies* the data values.
    self.n_preds = len(preds)

'> self.mvals = np.hstack(comp.dvals for comp in preds + unpreds)

..\model.py:434:

---

tup = <generator object at 0x000000000C8A4318>

def hstack(tup):
    """
    Stack arrays in sequence horizontally (column wise).

    Take a sequence of arrays and stack them horizontally to make
    a single array. Rebuild arrays divided by `hsplit`.

    Parameters
    ----------
    tup : sequence of ndarrays
        All arrays must have the same shape along all but the second axis.

    Returns
    -------
    stacked : ndarray
        The array formed by stacking the given arrays.

    See Also
    --------
    vstack : Stack arrays in sequence vertically (row wise).
    dstack : Stack arrays in sequence depth wise (along third axis).
    concatenate : Join a sequence of arrays together.
    hsplit : Split array along second axis.

    Notes
    -----
    Equivalent to ``np.concatenate(tup, axis=1)``

    Examples
    --------
    >>> a = np.array((1,2,3))
    >>> b = np.array((2,3,4))
    >>> np.hstack((a,b))
    array([1, 2, 3, 2, 3, 4])
    >>> a = np.array([[1],[2],[3]])
    >>> b = np.array([[2],[3],[4]])
    >>> np.hstack((a,b))
    array([[1, 2],
           [2, 3],
           [3, 4]])

    """

'> return _nx.concatenate(map(atleast_1d,tup),1)

....\numpy\core\shape_base.py:270:

---

.0 = <listiterator object at 0x000000000C89EEB8>

'> self.mvals = np.hstack(comp.dvals for comp in preds + unpreds)

..\model.py:434:

---

self = <xija.component.heat.SolarHeat object at 0x000000000C89C048>

@property
def dvals(self):
    if not hasattr(self, 'pitches'):
        self.pitches = self.pitch_comp.dvals
    if not hasattr(self, 't_days'):
        self.t_days = (self.pitch_comp.times
                       - DateTime(self.epoch).secs) / 86400.0
    if not hasattr(self, 't_phase'):
        time2000 = DateTime('2000:001:00:00:00').secs
        time2010 = DateTime('2010:001:00:00:00').secs
        secs_per_year = (time2010 - time2000) / 10.0
        t_year = (self.pitch_comp.times - time2000) / secs_per_year
        self.t_phase = t_year * 2 * np.pi

'> Ps = self.parvals[0:self.n_pitches] + self.bias

..\component\heat.py:116:

---

self = <xija.component.heat.SolarHeat object at 0x000000000C89C048>

def _func(self):

'> return self.pars[index].val
E IndexError: list index out of range

..\component\base.py:50: IndexError
==================================== 1 failed, 3 passed, 1 skipped in 0.38 seconds ====================================

[taldcroft]

The test failure looks like a configuration / version issue, since the code that is giving the IndexError at .\component\base.py:50 is from xija 0.2.7, not 0.3.2.

This test should definitely pass on all supported platforms since it is testing the main change in xija 0.3 to allow SolarHeat models with different numbers of pitch bins. There are currently no production models that use the capability (hence MCC passing), but I believe @matthewdahmer is planning on taking advantage of this for the next release of the MinusYZ or other models.

[taldcroft]

The Chandra.Time test ran fine on Windows (though I had to run it the way I normally do, through the setup.py utility due to stdin issues) - just the one difference in date representation that always occurs:

AssertionError: '01/01/99 12:13:14' != 'Fri Jan 1 12:13:14 1999'

@mbaski - Agreed we can let this go for now but it may be possible to fix the test. On windows can you do the following:

import datetime
d = datetime.datetime(1999, 1, 1, 12, 13, 14)
d.ctime()

This should give the local OS representation of that datetime. I'm guessing on Windows you'll see 01/01/99 12:13:14.

[mbaski]

Windows actually generates : 'Fri Jan 1 12:13:14 1999' for that. It's one of the mxDateTime tests that generates a difference:

from Chandra.Time import DateTime
mxd = DateTime('1999-01-01 12:13:14').mxDateTime
DateTime(mxd).mxDateTime.strftime('%c')

Outputs '01/01/99 12:13:14' on Windows, but ''Fri Jan 1 12:13:14 1999'' on linux.

[taldcroft]

Rats, I was hoping the mxDateTime %c directive was just the same as datetime.ctime() so there could be an OS-agnostic test. I'll just chop out that test in the next version since it's not really relevant for Chandra work. Using mxDateTime is deprecated anyway, I regret putting that in there a long time ago.

[mbaski]

Thanks for catching the configuration issue I had with Xija. For some reason, setup.py installed some of the updates but not all. I had to completely delete the xija directory and then re-run the install to get all the updates. The test appears to be successful now (4 passed, but 1 skipped?):

================================================= test session starts =================================================
platform win32 -- Python 2.7.2 -- pytest-2.2.4
collected 5 items

test_models.py s....

========================================= 4 passed, 1 skipped in 0.18 seconds =========================================

[taldcroft]

Woohoo for unit tests. :-)

The one skip is expected for a platform without direct access to the commanded states database.