README.md

enterprise-components general documentation

To start with enterprise-components, we recommend going through following materials:

Tutorial

• Installation
• Troubleshooting for Linux and Windows
• Lesson 1 - basic system
• Lesson 2 - adding quote table
• Lesson 3 - storing data in hdb
• Lesson 4 - user queries
• Lesson 5 - authorization and authentication

DemoSystem

The tutorial is based on the DemoSystem. Most recent DemoSystem package can be downloaded from releases section. The package for the system deployment consists of fully prepared:

• bin directory with all the components, libraries and yak binaries
• etc directory with configuration for the system and env.sh (and env.bat for Windows) script with environmental settings

ec elements

Components and libraries that are essential part of the ec:

• The components support the efficient construction of kdb+ infrastructure by providing the following functionality:
  • Data feeding
    • feedCsv - generic CSV files reader and publisher
  • Data distribution
    • tickHF - Publishing and distribution of High Frequency data
    • tickLF - Distribution of Low Frequency data
  • Data processing and storage
    • rdb - In-memory database
    • hdb - Historical database
    • hdbWriter - Hdb data upload
    • eodMng - End-of-day processing
    • rproc - Realtime processing
    • stream - Stream-based data processing
  • Data access
    • accessPoint - End users entry point
  • System maintenance
    • yak - Process Management Tool
    • hk - Housekeeping
  • Monitor server
    • monitor - Server monitoring tool
  • Testing
    • qtest - Test framework
    • mock - Set of mocks
• [Libraries] (../libraries)
  • qsl - q standard library - set of common libraries used across the system
    • sl - Standard library - common frame for all ec q scripts
    • pe - Protected evaluation library - wrapper for invoking functions in protected evaluation
    • event - Event library - execution of q functions as events
    • callback - Callback library - managing callbacks
    • timer - Timer library - abstraction layer for timer
    • handle - Connection management library - abstraction layer for interprocess connections
    • authorization - Authorization library - access restrictions and auditing
    • u - u.q library - extension of u.q from Kx
    • sub - Subscription management library - abstraction for realtime data subscription
    • store - Store library - data storage in the hdb
    • os - Os library - shell commands abstraction, covers Linux, MacOS and Windows
    • parseq - Parseq library - Parseq is a Q clone of the Haskell's Parsec, a parser combinator library.
    • tabs - Data model validation - set of functions facilitating data model validation

Articles

Set of articles describing various aspects of ec usage:

• Configuration:
• General configuration concept
• Configuration system.cfg
• Configuration dataflow.cfg
• Configuration access.cfg
• Configuration sync.cfg
• Elements of the configuration file
• Understanding cfg and qsd files correlation
• Misc:
• Security model description
• Logging concept
• Setup system in UTC or local time
• Testing the end-of-day procedure
• Development:
• Exxeleron q coding conventions - brief introduction to coding guidelines
• Documenting q code
• Document generation