Alice Engineering Comms 🪬 #1406

johnandersen777 · 2022-07-18T16:15:07Z

johnandersen777
Jul 18, 2022

These are the engineering logs of entities working on Alice. If you
work on Alice please use this thread as a way to communicate to others
what you are working on. Each day has a log entry. Comment with your
thoughts, activities, planning, etc. related to the development of
Alice, our open source artificial general intelligence.

This thread is used as a communication mechanism for engineers so that
others can have full context around why entities did what they did
during their development process. This development lifecycle data helps
us understand more about why decisions were made when we re-read the
code in the future (via cross referencing commit dates with dates in
engineering logs). In this way we facilitate communication across
both time and space! Simply by writing things down. We live an an
asynchronous world. Let's communicate like it.

We are collaboratively documenting strategy and implementation as
living documentation to help community communicate amongst itself
and facilitate sync with potential users / other communities /
aligned workstreams.

Game plan
- Move fast, fix things. Live off the land. Failure is not an option.
References
- This thread in git grep -i friendly format: https://github.com/intel/dffml/tree/main/docs/discussions/alice_engineering_comms
- Video: General 5 minute intro to Alice
- https://github.com/intel/dffml/blob/main/docs/tutorials/rolling_alice/0000_forward.md
- https://github.com/intel/dffml/blob/main/docs/tutorials/rolling_alice/0000_preface.md
- https://gist.github.com/07b8c7b4a9e05579921aa3cc8aed4866
  - Progress Report Transcripts
- https://github.com/intel/dffml/tree/main/entities/alice/
- https://github.com/intel/dffml/tree/main/docs/tutorials/rolling_alice/0000_architecting_alice#what-is-alice
- 2nd Party #1401
  - docs: arch: Inventory #1207
  - docs: arch: 2nd and 3rd party plugins #1061
- service: sw: src: change: notify: Service to facilitate poly repo pull model dev tooling #1315
- docs: examples: innersource: Org health and issue prioritization #1287
- Aligned threads elsewhere (in order of appearance)
  - @dffml
    - DFFML Weekly Sync Meeting Minutes
      - Alice isn't mentioned here but the 2nd party work is, Alice will be our maintainer who helps us with the 2nd party ecosystem.
    - Alice #1369
  - @pdxjohnny
    - https://twitter.com/pdxjohnny/status/1522345950013845504
    - https://mastodon.social/@pdxjohnny/109320563491316354
- Google Drive: AliceIsHere
  - Supplementary resources, status update slide decks, cards (with instructions of how to get more printed), Entity Analysis Trinity drawio files, screenshots. Miscellaneous other files.
- async comms / asynchronous communications
  - https://twitter.com/SergioRocks/status/1579110239408095232

Engineering Log Process

Alice, every day at 7 AM in Portland's timezone create a system context (the tick)
- Merge with existing system context looked up from querying this thread if exists
  - In the future we will Alice will create and update system contexts.
    - We'll start with each day, then move to a week, then a fortnight, then 2 fortnights.
    - She'll parse the markdown document to rebuild the system context as if it's cached
      right before it would be synthesized to markdown, we then run updates and trigger
      update of the comment body. Eventually we won't use GitHub and just DID based stuff.
      We'll treat these all as trains of thought / chains of system contexts / state of the
      art fields.
      - Take a set of system contexts as training data
      - The system context which we visualize as a line dropped from the peak of a pyramid
        where it falls through the base.
      - We use cross domain conceptual mapping to align system contexts in a similar direction
        and drop ones which do are unhelpful, do not make the classification for "good"
      - What remains from our circular graph is a pyramid with the correct decisions
        (per prioritizer)
      - This line represents the "state of the art", the remembered (direct lookup) or
        predicted/inferred system contexts along this line are well rounded examples of
        where the field is headed, per upstream and overlay defined strategic plans
        and strategic principles
        
        References:
        
        $ git grep -C 5 -i principle -- docs/arch/
        
        Source: Alice #1369
- Inputs
  - date
    - Type: Union[str, Date]
    - Example: 2022-07-18
    - Default: did:oa:architype:current-date
- Add yesterdays unfinished TODO items to this train of though with the
- Create a document (docutils?)
  - Make the top level header date with "Notes" appended
  - Collect all previous days TODOs from within the individual entity comments within the thread for the days comment (the team summary for that day)
    - Drop any completed or struck through TODOs
    - Output a list item "TODO" with the underlying bullets with handle prepended and then the TODO contents
      - Create comments for individuals after this the current system context is posted and we have a live handle to it to reply with each individuals markdown document.
- Synthesis the document to markdown (there is a python lib out there that can do docutils to md, can't remember the name right now)
- Upsert comment in thread
Any time (it's a game)
- We find something we need to complete Alice #1369 that was just completed by someone else
  - Paste ![chaos-for-the-chaos-god](https://github.com/intel/dffml/assets/5950433/636969a1-1f0f-4c96-8812-f10fa403e79c)
  - chaos: diversity of authorship applied to clustering model run on all thoughts (dataflows) where each dataflow maps to line changed and the cluster classification is the author.
    - service: sw: src: change: notify: Service to facilitate poly repo pull model dev tooling #1315 (comment)
- We accelerate a train of thought
  - Paste 🛤️
- We use manifests or containers
  - Paste all the things meme ![oci-all-the-things](https://user-images.githubusercontent.com/5950433/222979759-0dd374b2-ee5f-4cbc-92d1-5cb8de078ee8.png)
- We find more ways to on-ramp data to the hypergraph
  - ![knowledge-graphs-for-the-knowledge-god](https://user-images.githubusercontent.com/5950433/222981558-0b50593a-c83f-4c6c-9aff-1b553403eac7.png)
- We do a live demo
  - ![live-demo-for-the-live-demo-god](https://user-images.githubusercontent.com/5950433/226699339-45b82b38-a7fc-4f2f-a858-e52ee5a6983d.png)
- We see alignment happening
  - ![such-alignment](https://user-images.githubusercontent.com/5950433/226707682-cfa8dbff-0908-4a34-8540-de729c62512f.png)
- We enable bisection or hermetic or cacheable builds
  - ![hash-values-everywhere](https://user-images.githubusercontent.com/5950433/230648803-c0765d60-bf9a-474a-b67e-4b4177dcb15c.png)

Source: unknown

johnandersen777 · 2022-07-18T16:28:51Z

johnandersen777
Jul 18, 2022
Author

2022-07-18 Engineering Logs

TODO
- @aliceoa, @pdxjohnny: Kick off OSS scans
  - Targeting collaboration with CRob on metrics insertion to OpenSSF DB
- @pdxjohnny: Finish Q3 plans (Gantt chart, meeting templates, etc.)
  - Generate template for auto creation to fill every meeting / fillable pre-meeting

1 reply

johnandersen777 Jul 18, 2022
Author

2022-07-18 @pdxjohnny Engineering Logs

TODO
- Kick off OSS scans
  - Targeting collaboration with CRob on metrics insertion to OpenSSF DB
- Finish Q3 plans (Gantt chart, meeting templates, etc.)
  - Generate template for auto creation to fill every meeting / fillable pre-meeting
Future
- Engage with Loihi community
  - See what we can do here, not sure yet, play with system context / mitigation inference in devcloud?
  - https://www.intel.com/content/www/us/en/research/neuromorphic-community.html
  - https://download.intel.com/newsroom/2021/new-technologies/neuromorphic-computing-loihi-2-brief.pdf
  - https://www.intel.com/content/www/us/en/newsroom/news/intel-unveils-neuromorphic-loihi-2-lava-software.html
References
- https://medium.com/51nodes/decentralized-schema-registry-aa662b8db12b
- https://www.microsoft.com/security/blog/2021/10/06/microsofts-5-guiding-principles-for-decentralized-identities/
- https://ariadne.space/2022/07/17/how-efficient-can-cat1-be/
  - Usage of splice
- https://github.com/NVlabs/eg3d
  - Seeing from different perspectives, inter domain conceptual mapping, encoded sysctxs alternate mitigations
- https://github.com/robmarkcole/satellite-image-deep-learning
  - Knitting together system contexts (Alice could use for integration of various architectures)

johnandersen777 · 2022-07-19T15:38:19Z

johnandersen777
Jul 19, 2022
Author

2022-07-19 Engineering Logs

TODO
- @aliceoa, @pdxjohnny: Kick off OSS scans
  - Targeting collaboration with CRob on metrics insertion to OpenSSF DB
- @pdxjohnny: Finish Q3 plans (Gantt chart, meeting templates, etc.)
  - Generate template for auto creation to fill every meeting / fillable pre-meeting

1 reply

johnandersen777 Mar 7, 2023
Author

2022-07-19 @pdxjohnny Engineering Logs

TODO
- Kick off OSS scans
  - Targeting collaboration with CRob on metrics insertion to OpenSSF DB
- Finish Q3 plans (Gantt chart, meeting templates, etc.)
  - Generate template for auto creation to fill every meeting / fillable pre-meeting
- Follow up with OneAPI folks
- Overlay to alice shouldi contribute to create git repos when found from forks of PyPi packages
  - Associated tutorial
    - Linked from README
- Finish out alice please contribute recommended community standards
  dynamic opimp for meta issue body creation
  - Associated tutorial
    - Linked from README and CONTRIBUTING
Some good spdx DAG stuff on how we turn source into build SBOM wise
- https://lists.spdx.org/g/Spdx-tech/message/4659
References
- https://github.com/nsmith5/rekor-sidekick
  - Rekor transparency log monitoring and alerting
  - Leverages Open Policy Agent
  - Found while looking at Open Policy Agent to see if we can serialize to JSON.
  - Possibly use to facilitate our downstream validation
    - service: sw: src: change: notify: Service to facilitate poly repo pull model dev tooling #1315
- https://mermaid-js.github.io/mermaid/#/c4c
  - Mermaid is working on native https://c4model.com support!
- W3C approves DIDs!

"Intel Corporation congratulates the DID Working Group on Decentralized Identifier (DID) 1.0 reaching W3C Recommendation status.

DID provides a framework to unify and consolidate multiple evolving identity systems. Consolidating identity systems within a single framework is useful for validating the authenticity of information and preserving its integrity as it is moved and processed among cloud, edge, and client systems. This potentially increases the capabilities of the Web to connect and unify multiple sources of information.

The continuing evolution of this work will be key to the development of new technologies in the fields of supply chain management and Internet of Things (IoT) devices and services. For example, a Birds of a Feather (BOF) discussion group at IETF Supply Chain Integrity, Transparency, and Trust (SCITT) has already highlighted DID as a useful approach in providing much needed structure for exchanging information through the supply chain, and the Web of Things (WoT) WG is planning to support DID for identifying and discovering IoT devices and metadata.

Intel Corporation supports this work and encourages the DID Working Group to continue working towards the convergence of widely implemented and adopted standardized best practices for identity in its next charter."

Eric Siow, Web Standards and Ecosystem Strategies Director, Intel Corporation

https://blog.devgenius.io/top-10-architecture-characteristics-non-functional-requirements-with-cheatsheat-7ad14bbb0a9b

For Vol 3, on mind control
- https://bigthink.com/the-present/sophists/

Unsent to Mike Scovetta: michael.scovetta (at) microsoft.com

Hi Mike,

Hope you’ve been well. It’s John from Intel. Thanks again to you and the team for welcoming me to the Identifying Security Threats working group meeting 2021-02-18 last year. We talked a bit about how Intel had a similar effort. I then changed roles hoping to get more involved with OpenSSF but then ended up getting told to be uninvolved. Now I switched roles again and involvement is in scope! Sorry for the lapse in communications.

I periodically check the minutes so I joined today and asked about the "Alpha-Omega" project from last week’s minutes which I then did some research on. We just started what looks to me to be an aligned project, coincidentally named Alice Omega Alpha: https://github.com/intel/dffml/tree/alice/entities/alice

Alice is the nickname for both the entity and the architecture, the Open Architecture, which is a methodology for interpretation of existing well established, formats, protocols, and other domain specific representations of architecture. What we end up with is some JSON, YAML, or other blob of structured data that we can use to build cross language tooling focused more on policy and intent, incorporating data from arbitrary sources to create a holistic picture of software across dependency boundaries by focusing on threat models.

Alice will be doing scans of open source projects and we’d still love to collaborate to contribute metrics to the OpenSSF metrics database, we can easily have her shoot applicable metrics off to that DB. We’ve also been looking at fusing VEX and DIDs to facilitate distributed vulnerability disclosure and patch distribution.

Unset to Jun Takei: jun.takei (at) intel.com

The W3C today issued the recommendation on DIDs. Jun I saw from Eric's
comment on the press release that the SCITT working group has an SCITT
Architecture which DID's might be suitable for.

The DFFML community is working on a project called Alice
https://github.com/intel/dffml/tree/alice/docs/tutorials/rolling_alice
she is intended to be a developer helper. She's also the way we data mine
source repositories (etc.).

She’s open source with a plugin system ("overlays") so we can write open source code
and then just add our internal integrations. This system relies on an abstraction of
architecture known as the Open Architecture. The Open Architecture, also known as
Alice, is a methodology for interpreting directed graphs of domain specific architectures.
Alice is the name we give both the entity and the architecture. We are hoping to
have Alice store and process information backed by directed graphs of DIDs, SBOMs, and
VEX info primarily. This sounds very similar to the SCITT Architecture. We would love to
collaborate with you both to help make SCITT a success. Alice is focused on analysis of
our software supply chain so as to ensure we conform to best practices. We would like
the analysis to serialize directly to an industry best practice format for that as well,
which SCITT looks to be.

To increase the level of trust in our supply chain we would like to ensure interoperability
up and down the stack. Ned is involved in the DICE space and communicated to me
that

Please let us know where things are at with your involvement with DIDs and SCITT so we
can be in sync with Intel's involvement and direction in this space. Please also let us know
how we could best establish an ongoing line of communication so as to build off and
contribute to where possible the work you're involved in.

References:

Unsent

To: Jun and Mike and Dan lorenc.d (at) gmail.com

I commented on the OpenSFF Stream 8 doc recommending that DIDs be looked at
as a way to exchange vulnerability information.

We've been looking potentially at a hybrid DID plus rekor
architecture (DIDs eventually as a proxy to)

References:

https://github.com/sigstore/rekor

johnandersen777 · 2022-07-20T14:49:37Z

johnandersen777
Jul 20, 2022
Author

2022-07-20 Engineering Logs

TODO
- @aliceoa, @pdxjohnny: Kick off OSS scans
  - Targeting collaboration with CRob on metrics insertion to OpenSSF DB
- @pdxjohnny: Finish Q3 plans (Gantt chart, meeting templates, etc.)
  - Generate template for auto creation to fill every meeting / fillable pre-meeting
- @dffml: Get involved in SCITT
  - Meetings
    - https://docs.google.com/document/d/1vf-EliXByhg5HZfgVbTqZhfaJFCmvMdQuZ4tC-Eq6wg/edit#
    - Weekly Monday at 8 AM Pacific
    - https://armltd.zoom.us/j/99133885299?pwd=b0w4aGorRkpjL3ZHa2NPSmRiNHpXUT09
  - Mailing list
    - https://www.ietf.org/mailman/listinfo/scitt
    - https://mailarchive.ietf.org/arch/browse/scitt/
  - Slack
    - https://mailarchive.ietf.org/arch/msg/scitt/PbvoKOX996cNHJEOrjReaNlum64/
    - Going to email Orie Steele orie (at) transmute.industries to ask for an invite.

2 replies

johnandersen777 Jul 20, 2022
Author

2022-07-20 Identifying Security Threats WG

Mike leading
Marta
- Office hours
  - Place for Open Source maintainers to be able to ask community of security experts
  - Idea is to run first two sessions in August and September
  - Proposing two different timeslots to cover all geos
  - Question is will be be well staffed for both of those
  - She is collecting feedback right now on possibilities for those dates
  - Pinging folks who have shown interest in the past
  - What format?
    - People just show up and ask
    - Registration with topic they want to talk about
      - Allows us to prepare, consensus is currently we like this
      - Can grab right experts beforehand this way
  - Reaching out to logistics team for how we can communicate
    zoom links, etc.
  - Will test registration beginning of August
  - Will do doodle poll or something for slots
  - Jen is the one for the Zoom setup
  - Amir from ostif.org volunteering to answer questions
  - May want to do a blog in addition to twitter
  - Outreach maybe 4th or 5th, have the twitter points back
    to the blog to capture that.
Meeting time update
- We have been doing this at this time for about a year or so
- We previously alternated between two timeslots for Europe and Asia
- Should we keep this 10 AM Pacific timeslot?
  - Alternate between US and APAC friendly timezone
- Most other WGs are morning Pacific time
Technical Advisory Committee (TAC) update
- They are tasked with making sure we are delivering on our
  cohesive promise, part of that is visuabliity and transparency
  into the work that we do.
- We now have a formal reporting process
- It's not a periodic we're all invited to show up to the TAC meeting
  one slide per project.
  - What we're doing
  - Why we're doing it
- It's meant as an FYI, we are not asking for approval, we're letting them
  know what we're up to.
- Everyone who is driving a process or project or thing, please send Mike
  a single slide, what is it, why are we doing it, what the status is,
  what's coming next, and if you need anything
- Christine on metrics
- Luigi for SECURITY-INSIGHTS.yml`
- Mike will send out a template
  - Please fill and respond by Monday
- Mike says the metrics work should live under a working group, maybe this one, maybe best practices
- CRob might have an opinion here, as long as work gets done
- As an org OpenSSF would benefit by being less siloed
- Question on if we should align to streams?
- LFX specific definition of metrics in mobilization paper
- AR for Christine to sync with CRob and see what he thinks.
- Will raise with TAC next week.
A few action items for metrics from Christine
- Working groups are adopting streams from the mobilization plans
Mike: Alpha Omega
- A few people were on the public call earlier
- The recording will be on YouTube
- Mike will give the fast version of the presentation right now
- They are still hiring
- Exploring ways of allocating headcount other than direct hiring
- If you know anyone or are interested please apply or ping them!
- Alpha
  - Announced Node, Python, Eclipse
- Omega
  - Toolchain is pending
  - Waiting for legal approval due to the way the license for CodeQL works
  - Had a CVE in Node that got fixed earlier this month
  - RCE in JSHint that was bitrotted (unused) we removed
  - Two CVEs discloudsed yetserday and two more in the works (couple weeks to release ETA)
  - Found NodeJS vuln via systemcall tracing
    - It tires to query openssl.cnf and dumps strace logs to a repo
    - You then have a one stop show of show me every link package of when a binary starts, it does a DNS query
      - John: Sounds aligned with Alice's goals
- https://sos.dev coming under Alpha-Omega
  - Allows us to compensate dev directly
- How to participate
  - Improve security tools
  - https://sos.dev
  - Join working groups
  - Get on slack
Amir: Security Reviews
- Repo is looking good
- Updating with four new audits that ostif.org published last week
- At almost 100 reviews from Mike (Omega work), ostif.org, and community
- We're gaining traction, getting good stuff in there all the time
- Might need some help with the automated testing that get's done
  when we upload reviews.
- Feedback always welcome.
John: Collection of metric / Alpha-Omega data into shared DB
- https://github.com/intel/dffml/tree/alice/docs/tutorials/rolling_alice
- https://datatracker.ietf.org/doc/html/draft-birkholz-scitt-architecture
- https://www.w3.org/2022/07/pressrelease-did-rec.html.en
- https://docs.microsoft.com/en-us/azure/confidential-ledger/architecture
- Mike
  - Mike has been thinking about SCITT as a schema and rules on how one would assert facts, weither it's confidential compute or traditional permissions is impelmenetation details.
  - If metircs runs across you're repo and you have 30 contributors, great
  - As consumer, how can I discover that fact and trust that it's accruate
  - Could immaiget a world where things like Scorecard express the data as as SCITT assursion
  - You go and query that store and you say tell me everythig you know about foo and you get it all back
  - Until we have an implementation with WEb5 that's at at least beta, we could expore what that looks like.
    - John: We can do rekor for now, we'll bridge it all later target 1-2 years out
    - John: We have alignment. Time to execute. rekor + sigstore for metric data atteststation signed with github odic tokens. We care about data provenance. We will later bridge into web5 space used as central points of comms given DID as effectively the URL or the future. This is in realtion to what we talked to Melvin about with data provenance. We need to start planning how we are going to build up this space now so we can have provenance on thoughts later. This provenance could be for example on inference derived from provenance from training data and model training env and config. This will allow us to ensure the prioritizer make decisions based on Sprit of the law / aka intent based policy derived from Trinity of Static Analysis, Dynamic Analysis, and Human Intent.
      - Living Threat Model threats, mitigations, trust boundaries as initial data set for cross domain conceptual mapping of the the trinity to build pyramid of thought alignment to strategic principles.
      - One of our strategic plans / principles says: "We must be able to trust the sources of all input data used for all model training was done from research studies with these ethical certifications"
        
        This allows us to write policies (Open Policy Agent to JSON to DID/VC/SCITT translation/application exploration still in progress) for the organizations we form and apply them as overlays to flows we execute where context appropriate. These overlaid flows define the trusted parties within that context as applicable to the active organizational policies as applicable to the top level system context.
        
        The policy associated with the principle that consumes the overlaid trust attestations we will implement and LTM auditor for which checks the SCITT provenance information associated with the operation implementations and the operation implementation network, input network, etc. within the orchestrators trust boundary (TODO need to track usages / reuse of contexts ictx, nctx, etc. with something predeclared, aka at runtime if your Operation data structure doesn't allowlist your usage of it you can pass it to a subflow for reuse. This allows us to use the format within our orchrestration and for static analysis because we can use this same format to describe the trust boundry proeprties that other domain sepcific represenatations of architecture have, for instance we could if we were doing and Open Architecture (OA) Intermediate Representation (IR) for and ELF file we might note that the input network context is not reused from the top level system context. Where as if we did an OA IR for Python code we would say that the input network is reused from the top level system context (it has access to that memory region, whereas when you launch and ELF you look access to the parents memory region, typically).
  - Christine
    - Looking at trying to connect all the different data sources
References
TODO
- @pdxjohnny
  - Reach out to Christine about metrics collaboration
  - Respond with slides for Mike if he asks

johnandersen777 · 2022-07-21T22:04:10Z

johnandersen777
Jul 21, 2022
Author

2022-07-21 Engineering Logs

https://docs.rs/differential-dataflow/latest/differential_dataflow/
https://lists.spdx.org/g/Spdx-tech/message/4673
- It is not just a matter of your software, it is a fundamental design question whether to maintain separation between the logical model and its serializations. Maintaining separation shouldn't be a matter of personal preference, it's good software engineering. The OWL Web Ontology Language https://www.w3.org/TR/owl2-overview/ has an excellent diagram illustrating the separation between semantics and syntax. Several serializations are defined in OWL (Manchester Syntax, Functional Syntax, RDF/XML, OWL/XML, and Turtle), and more syntaxes have been added since (JSON-LD, RDF-star, ...).

0 replies

johnandersen777 · 2022-07-23T17:54:28Z

johnandersen777
Jul 23, 2022
Author

2022-07-23

https://blog.ciaranmcnulty.com/2022-05-12-multiple-build-contexts

0 replies

johnandersen777 · 2022-07-24T15:54:29Z

johnandersen777
Jul 24, 2022
Author

2022-07-28 Alice Intelligence/Open Architecture Working Group Initial Meeting

Meeting info
- 8-9 AM Pacific
- https://meet.google.com/kox-ssqn-kjd

0 replies

johnandersen777 · 2022-07-25T14:50:20Z

johnandersen777
Jul 25, 2022
Author

2022-07-25 Engineering Logs

TODO
- @aliceoa, @pdxjohnny: Kick off OSS scans
  - Targeting collaboration with CRob on metrics insertion to OpenSSF DB
- @pdxjohnny: Finish Q3 plans (Gantt chart, meeting templates, etc.)
  - Generate template for auto creation to fill every meeting / fillable pre-meeting
- @dffml: Get involved in SCITT
  - Meetings
    - https://docs.google.com/document/d/1vf-EliXByhg5HZfgVbTqZhfaJFCmvMdQuZ4tC-Eq6wg/edit#
    - Weekly Monday at 8 AM Pacific
    - https://armltd.zoom.us/j/99133885299?pwd=b0w4aGorRkpjL3ZHa2NPSmRiNHpXUT09
  - Mailing list
    - https://www.ietf.org/mailman/listinfo/scitt
    - https://mailarchive.ietf.org/arch/browse/scitt/
  - Slack
    - https://mailarchive.ietf.org/arch/msg/scitt/PbvoKOX996cNHJEOrjReaNlum64/
    - Going to email Orie Steele orie (at) transmute.industries to ask for an invite.

2 replies

johnandersen777 Jul 25, 2022
Author

2022-07-25 Supply Chain Integrity, Transparency and Trust (SCITT)

TODO
- Get involved in SCITT
  - Meetings
    - https://docs.google.com/document/d/1vf-EliXByhg5HZfgVbTqZhfaJFCmvMdQuZ4tC-Eq6wg/edit#
    - Weekly Monday at 8 AM Pacific
      - Joining today
    - https://armltd.zoom.us/j/99133885299?pwd=b0w4aGorRkpjL3ZHa2NPSmRiNHpXUT09
  - Mailing list
    - https://www.ietf.org/mailman/listinfo/scitt
    - https://mailarchive.ietf.org/arch/browse/scitt/
  - Slack
    - https://mailarchive.ietf.org/arch/msg/scitt/PbvoKOX996cNHJEOrjReaNlum64/
    - Going to email Orie Steele orie (at) transmute.industries to ask for an invite.
Links
Folks will be in Philly on Thursday for meeting
- There is a remote link (see mailing list?) for the Thursday meeting
Typical global warming centric chit chat
Others at RATs meeting or busy with other IETF activities
Introductions
- Kelvin Cusack
  - Filling in for John from his org
- John Andersen
  - Connecting dots between this and OpenSSF
    - Yogesh was excited to see someone from Intel
      - Intel is involved in RATs but not as much here
- Kiran Karunakaran
  - Microsoft
  - On Kay Williams's team
  - Will likely lead this meeting in the future
Upcoming Birds of Feather (BoF)
- You need to register here: https://docs.google.com/document/d/1vf-EliXByhg5HZfgVbTqZhfaJFCmvMdQuZ4tC-Eq6wg/edit?pli=1#heading=h.214jg0n2xjhp
  - There is registration for remote
- Problem Statement currently scoped around software
- We went back to a more scoped problem statement is that we want to form a formal working group in the IETF for SCITT.
- In order to form it we have to have stuffiest people show interest in the problem space
- Need problem space and charter so that it's scoped enough that the leadership is confident that the group can make progress
- The early design proposal for the SCITT transparency service is that the service is content agnostic
- any kind of metadata could be provided and retrieved
- SBOMs, Software Test Cases, Hardware BOMs, Test results on other types of products
- Electronic ballots (Ray, see mailing list), oil, gas, physical goods (U.S. govt.)
- In order to gain confidence from the leadership at IETF to form the WG we felt it was critical to narrow the scope for now to software
  - Leadership thought scope was too big at first
Thoughts around scope
- Charter is focused on software, an attainable goal
- Once we have a WG we can later broaden the scope via re-chartering
We will design something that works for hardware and for software
- We are hanging software window curtains but we are looking at everything
Software systems interact with everything else
- Dick Brooks (REA), any manifest could be signed and processes with SCITT
- It's just metadata, what was issued, how it was issued
  - @dffml: We are encoding system contexts, Alice, into the chain, one place where she will live.
Opens
- Open Policy Agent (mentioned in meeting minutes doc future topics)
  - What are your plans / thoughts around Open Policy Agent and encoding policies into SCITT?
    - Policy can be used in two places
      - Policy for what can be put onto register
        
        Some registries might constrain themselves for what types of data they allow
    - Policy for someone evaluating the contents of the registry to make discussions for fitness of use
  - REGO also considered as a policy language
    - Perhaps decide on multi
    - This policy discussion will happen in this WG for now, then maybe a sub working group
- Dick Brooks: Mentions HR4081
  - On topic for what we are
  - Talked about attestations for devices containing a camera and microphone and is connected to the internet
    - There will need to be an attestation from the device
      - Dick submitted to local rep to include software attestations as well
    - https://www.congress.gov/bill/117th-congress/house-bill/4081
      https://www.congress.gov/bill/117th-congress/house-bill/4081/text
Producers and other parties provide content into the system, attestations, claims, recorded into SCITT ledger
Dick contacted his congress person to ask to add an amendment to HR4081
- Amendment for smartphone apps to provide a trust score
  - Tie in with OpenSSF metrics database to grab the security of repos involved
- Dick
  - Proposed amendment I mentioned for HR 4081:
  - "Require smart phone app stores to include a software supply chain trust score for each app". This gives consumers the ability to check trustworthiness before installing an app,
Ray: think about attestations is different than the transparency ledger
- Thinks it's a lot to bite off to do both
- Are there IoT folks that might have more attestation experience we could tap into?
- Is there a sub-working group focused on device attestations (in response to HR4081)
  - Device attestations could be recorded tin the transparency ledger
  - TCG DICE WG is a target point of engagement (UCAN, CBOR, DID).
    - https://trustedcomputinggroup.org/work-groups/dice-architectures/
    - https://www.trustedcomputinggroup.org/wp-content/uploads/Device-Identifier-Composition-Engine-Rev69_Public-Review.pdf
    - Looking at hardware actively attesting
    - Microsoft has an Open Source implementation on GitHub
  - This attestation stuff starts to look at real life commerce, Ray thinks it's important to
Joshua Lock
- On software attestations, I have been working on a page for the SLSA website to describe the model we're working with. I can share to the SCITT list once the change is merged.
- IIRC the SCITT draft standards refer to a 'software attestation" as a "claim", to disambiguate from RATS & TCG attestations
Remote Attestation and Device Attestation
- Embraced COSE and CBOR
  - Also in SCITT
  - Hopefully we converge on underlying formats for both in-toto style and remote attestation style attestations
There are also NIST attestations
Vuln information mentioned by Kay as possible content inserted into SCITT
- This is a goal of ours with our CVE Binary Tool engagement
- We also could encode SBOMs from the systems that built them, we could patch sigstore to insert into a SCITT ledger

johnandersen777 · 2022-07-26T13:09:49Z

johnandersen777
Jul 26, 2022
Author

2022-07-26 Engineering Logs

TODO
- @aliceoa, @pdxjohnny: Kick off OSS scans
  - Targeting collaboration with CRob on metrics insertion to OpenSSF DB
- @pdxjohnny: Finish Q3 plans (Gantt chart, meeting templates, etc.)
  - Generate template for auto creation to fill every meeting / fillable pre-meeting
- @dffml: Get involved in SCITT
  - Meetings
    - https://docs.google.com/document/d/1vf-EliXByhg5HZfgVbTqZhfaJFCmvMdQuZ4tC-Eq6wg/edit#
    - Weekly Monday at 8 AM Pacific
    - https://armltd.zoom.us/j/99133885299?pwd=b0w4aGorRkpjL3ZHa2NPSmRiNHpXUT09
  - Mailing list
    - https://www.ietf.org/mailman/listinfo/scitt
    - https://mailarchive.ietf.org/arch/browse/scitt/
  - Slack
    - https://mailarchive.ietf.org/arch/msg/scitt/PbvoKOX996cNHJEOrjReaNlum64/
    - Going to email Orie Steele orie (at) transmute.industries to ask for an invite.

1 reply

johnandersen777 · 2022-07-27T20:43:28Z

johnandersen777
Jul 27, 2022
Author

2022-07-27 Engineering Logs

References
- kaniko coder k3d digitalocean
  - The following were issues with kind which might also effect us
    - Multi-stage builds fail when run in kind GoogleContainerTools/kaniko#2164
    - tektoncd/pipeline@6542823
  - It doesn't look like kaniko currently supports multi context builds
- Great example of communication and meeting procedures link to code
  - https://lists.spdx.org/g/Spdx-tech/message/4699

1 reply

johnandersen777 Jul 28, 2022
Author

2022-07-27 @pdxjohnny Engineering Logs

TODO
- Get involved in SCITT
  - Meetings
    - https://docs.google.com/document/d/1vf-EliXByhg5HZfgVbTqZhfaJFCmvMdQuZ4tC-Eq6wg/edit#
    - Weekly Monday at 8 AM Pacific
      - Joining today
    - https://armltd.zoom.us/j/99133885299?pwd=b0w4aGorRkpjL3ZHa2NPSmRiNHpXUT09
  - Mailing list
    - https://www.ietf.org/mailman/listinfo/scitt
    - https://mailarchive.ietf.org/arch/browse/scitt/
  - Slack
    - https://mailarchive.ietf.org/arch/msg/scitt/PbvoKOX996cNHJEOrjReaNlum64/
    - Going to email Orie Steele orie (at) transmute.industries to ask for an invite.
- Kick off OSS scans
  - Targeting collaboration with CRob on metrics insertion to OpenSSF DB
- Finish Q3 plans (Gantt chart, meeting templates, etc.)
  - Generate template for auto creation to fill every meeting / fillable pre-meeting
- Follow up with OneAPI folks
- Overlay to alice shouldi contribute to create git repos when found from forks of PyPi packages
  - Associated tutorial
    - Linked from README
- Finish out alice please contribute recommended community standards
  dynamic opimp for meta issue body creation
  - Associated tutorial
    - Linked from README and CONTRIBUTING
- Software Analysis Trinity diagram showing Human Intent, Static Analysis, and Dynamic Analysis to represent the soul of the software / entity and the process taken to improve it.
  - SoftwareAnalysisTrinity.drawio.xml

Refactoring and Thinking About Locking of Repos for Contributions

Metadata
- Date: 2022-07-27 20:30 UTC -7
Saving this diff which was some work on dynamic application of overlay
so as to support fixup of the OpImp for meta_issue_body()'s inputs.
- We are going to table this for now for time reasons, but if someone
  wants to pick it up before @pdxjohnny is back in September, please
  give it a go (create an issue).
Noticed that we have an issue with adding new files and locking. The current
lock is on the git_repository/GitRepoSpec.
- We then convert to AliceGitRepo, at which point anything take AliceGitRepo
alice
- Goal: Display Alice and software analysis trinity
- https://free-images.com/search/?q=alice%27s+adventures+in+wonderland&cat=st
- https://free-images.com/display/de_alices_abenteuer_im_43.html
- https://github.com/KhorSL/ASCII-ART
- Completed in d067273

$ python ascii_art.py /mnt/c/Users/Johnny/Downloads/alice-looking-up-white-background.png

$ alice
usage: alice [-h] [-log LOG] {please,shouldi,threats} ...

                            .,*&&888@@#&:,
                          .:&::,...,:&#@@@#:.
                         .o,.       ..:8@@#@@+
                        .8o+,+o*+*+,+:&#@@#8@@.
                        &8&###@#&..*:8#@@#@#@@&+.
                       ,@:#@##@@8,:&#@@@###@88@@.
                      ,#@8&#@@@#o:#@@@@#8#@#8+&#.
                     +8####@@@@###@@@888#@@@#oo#.
                   .*8@###@@@@@@@@@#o*#@@#@@#8o@,
                  +###@#o8&#@@##8::##@@@&&#@8#&+
                  o@8&#&##::.,o&+88#&8##8*@@#@#,
                 .##888&&oo#&o8###8&o##8##&####8,
                .&#@8&:+o+&@@@#8#&8:8@@@@@#8@@@oo+
               ,&&#@##oo+*:@###&#88,@@@@#@o&##&8#@o,.
              ,#&###@@8:*,#o&@@@@##:&#@###*.&o++o#@@#&+
              o8&8o8@#8+,,#.88#@#&@&&#@##++*&#o&&&#@@@@.
              *88:,#8&#,o+:+@&8#:8@8&8#@@&o++,*++*+:#@@*.
              .+#:o###@8o&8*@o&o8@o888@@@o+:o*&&,@#:&@@@,
                *+&@8&#@o#8+8*#+8#+88@@@@@@&@###8##@8:*,
                  +o.@##@@@&88@*8@:8@@@@@@:.. ,8@:++.
                    +&++8@@@@##@@@@@@@@@@@+    88
                    &.   *@8@:+##o&888#@@@,   .#+
                    &.   ,@+o,.::+*+*:&#&,    ,@.
                    &.   .@8*,. ,*+++.+*     :8+
                    :+   .#@::. .8:.:**    .8@@o,
                    .o.   #@+   :@,.&*   .:@@@@@@8**.
                     +&.  :@o,+.*o,*,  .*@@@@@@@@@@#o
                   .*:&o.  8@o:,*:,  .o@@#8&&@@@@#@@@*
                 ,*:+:::o.*&8+,++  ,&@@#:  * :@@88@@@#:.
               ,::**:o:.,&*+*8:  *8@@##o   *,.8@@#8#@#@#+
              *:+*&o8:. ,o,o:8@+o@@88:*@+  +: +#@#####8##&.
            ,:&::88&,   .&:#o#@@@#,+&&*#&. .:,.&#@#88#####&,
           +::o+&8:.    :##88@@@@:.:8o+&8&. .. +8###&8&##&88*
         .:*+*.8#:    ,o*.+&@@#@8,,o8*+8##+    .+#8##8&&#8888:.
        ,:o., &#8.  .:8*.  .o, &#,*:8:+,&*:,    .8@@#o&&##8:&#8.
      .*o.*,+o8#*  +8&,  .::. .88.+:8o: ,+:,    ,o#@#8&o8##&#8+
     +o, .+,,o#8+,8@o**.,o*,  :8o +*8#*  +&,    ,*o@@#@&8&oo8&:,
    oo*+,,,*8@#..&@8:**:oo+. +8#* *+#@:...oo+  .**:8@@@ooo&:&o##+
    ::+..,++#@,.:##o&o**,....oo#++#8#@:.,:8&:.....*&@@#:oo*&oo&#@*
     .+**:*8@o,+##&o:+,,,+,,o*8#,,8@#@:,,+*o*++,,,,+&#@8*8o88&::*.        .,,,,,++,
         ..8@++#@#88:,,,.,,,:+#&,,#@@#:,,.,&o*,.+++*:#@8+:*+.    ......,:+*&,,.....
            +:&8#@@##8&+,,,***@&,.8@@@*,,,.:o8&o&*o&o&o.     .,.****::*:o*:o*o+,.
                 ...,*:*o&&o*8@@&o8@@@8+,,+:&&:+,...     ,++*&oo&8&&&oo#@##8#&8:.
                        o@#@@@@#@@@@@@@,.....  ..,,.+*::o#@##@##@#@#########@@@8:,.
                        ,@##@@88#@@@@@8     .:***oo*#8###8#@#@#@#@####@#@###@@#8&#:
                         8+.,8+..,*o#@+     ,o+o88&88###@8#######@8#8#88#8#88##88#&
                         *o  *+     #8       .  ,*o&#@##@@@@@@@@@######8#888&&oo:8:
                          8, ,&    +@*        .ooo&#@@@@@#@@@@@@####@##8#8##oo:o&:,
                          +&  &,  .@#.        .:8#@@@@@@@@@@##8#####8#o&&#8*:8&&8:
                           o* ,o  o@&        +o#@@@@@@@@#o&o88:&+ooo&:*::o:o&**o.:*+
                           .8. 8.,o#8    .+&#@@@@@@@@&o+,::*+*:+:, ,. ,.. .,. ,.
                            8. 8.,.&@:*:&@@@@@@@@8o+,    ,.
                           :@o:#,,o8&:o&@@@@#&:+.
                          .@@@@@@@@@@@#8&o+,
                           ,*:&#@#&o*,..

                                /\
                               /  \
                              Intent
                             /      \
                            /        \
                           /          \
                          /            \
                         /              \
                        /  Alice is Here \
                       /                  \
                      /                    \
                     /______________________\

             Dynamic Analysis          Static Analysis

    Alice's source code: https://github.com/intel/dffml/tree/alice/entities/alice
    How we built Alice: https://github.com/intel/dffml/tree/alice/docs/tutorials/rolling_alice
    How to extend Alice: https://github.com/intel/dffml/blob/alice/entities/alice/CONTRIBUTING.rst
    Comment to get involved: https://github.com/intel/dffml/discussions/1406


positional arguments:
  {please,shouldi,threats}

optional arguments:
  -h, --help            show this help message and exit
  -log LOG              Logging Level

TODO
- Auto fork repo before push
  - Update origin to push to
  - Create PR
- Update README to fix demos
- Update CONTRIBUTING with tutorial on adding
  CONTRIBUTING.md check and contribution

entities/alice/alice/timelines.py

"""
Helpers for the timelines we support
"""

# Trinity Day 0
ALICE_DAY_0_GREGORIAN = datetime.datetime(2022, 4, 16)

def date_alice_from_gregorian(date: str) -> int:
    # TODO
    return ALICE_DAY_0_GREGORIAN

diff --git a/dffml/base.py b/dffml/base.py
index fea0ef7220..9d6cd886fa 100644
--- a/dffml/base.py
+++ b/dffml/base.py
@@ -237,6 +237,7 @@ def convert_value(arg, value, *, dataclass=None):
                         # before checking if the value is an instance of that
                         # type. Since it doesn't make sense to check if the
                         # value is an instance of something that's not a type.
+                        print(possible_type, value)
                         if isinstance(possible_type, type) and isinstance(
                             value, possible_type
                         ):
diff --git a/dffml/df/system_context/system_context.py b/dffml/df/system_context/system_context.py
index e055a343f1..063547ad0c 100644
--- a/dffml/df/system_context/system_context.py
+++ b/dffml/df/system_context/system_context.py
@@ -90,11 +90,11 @@ class SystemContextConfig:
     # links: 'SystemContextConfig'
     overlay: Union["SystemContextConfig", DataFlow] = field(
         "The overlay we will apply with any overlays to merge within it (see default overlay usage docs)",
-        default=APPLY_INSTALLED_OVERLAYS,
+        default=None,
     )
     orchestrator: Union["SystemContextConfig", BaseOrchestrator] = field(
         "The system context who's default flow will be used to produce an orchestrator which will be used to execute this system context including application of overlays",
-        default_factory=lambda: MemoryOrchestrator,
+        default=None,
     )
 
 
@@ -131,6 +131,7 @@ class SystemContext(BaseDataFlowFacilitatorObject):
         )
         # TODO(alice) Apply overlay
         if self.config.overlay not in (None, APPLY_INSTALLED_OVERLAYS):
+            print(self.config.overlay)
             breakpoint()
             raise NotImplementedError(
                 "Application of overlays within SystemContext class entry not yet supported"
diff --git a/dffml/high_level/dataflow.py b/dffml/high_level/dataflow.py
index d180b5c302..d595ae1cb4 100644
--- a/dffml/high_level/dataflow.py
+++ b/dffml/high_level/dataflow.py
@@ -206,12 +206,25 @@ async def run(
             # the of the one that got passed in and the overlay.
             if inspect.isclass(overlay):
                 overlay = overlay()
+            # TODO Move this into Overlay.load. Create a system context to
+            # execute the overlay if it is not already.
+            known_overlay_types = (DataFlow, SystemContext)
+            if not isinstance(overlay, known_overlay_types):
+                raise NotImplementedError(f"{overlay} is not a known type {known_overlay_types}")
+            if isinstance(overlay, DataFlow):
+                overlay = SystemContext(
+                    upstream=overlay,
+                )
             # TODO(alice) overlay.deployment("native.python.overlay.apply")
             apply_overlay = overlay.deployment()
             async for _ctx, result in apply_overlay(
                 dataflow=dataflow,
             ):
+                print("FEEDFACE", _ctx, result)
+                breakpoint()
+                return
                 continue
+
                 # TODO
                 resultant_system_context = SystemContext(
                     upstream=result["overlays_merged"], overlay=None,
diff --git a/dffml/overlay/overlay.py b/dffml/overlay/overlay.py
index 13a50d9c10..0a01d38de9 100644
--- a/dffml/overlay/overlay.py
+++ b/dffml/overlay/overlay.py
@@ -124,7 +124,7 @@ DFFML_MAIN_PACKAGE_OVERLAY = DataFlow(
             stage=Stage.OUTPUT,
             inputs={
                 "merged": DataFlowAfterOverlaysMerged,
-                "dataflow_we_are_applying_overlays_to_by_running_overlay_dataflow_and_passing_as_an_input": DataFlowWeAreApplyingOverlaysToByRunningOverlayDataflowAndPassingAsAnInput,
+                "upstream": DataFlowWeAreApplyingOverlaysToByRunningOverlayDataflowAndPassingAsAnInput,
             },
             outputs={"overlayed": DataFlowAfterOverlaysApplied,},
             multi_output=False,
@@ -208,15 +208,12 @@ merge_implementations(
 DFFML_OVERLAYS_INSTALLED.update(auto_flow=True)
 
 # Create Class for calling operations within the System Context as methods
-DFFMLOverlaysInstalled = SystemContext.subclass(
-    "DFFMLOverlaysInstalled",
-    {
-        "upstream": {"default_factory": lambda: DFFML_OVERLAYS_INSTALLED},
-        # TODO(alice) We'll need to make sure we have code to instantiate and
-        # instance of a class if only a class is given an not an instance.
-        "overlay": {"default_factory": lambda: None},
-        "orchestrator": {"default_factory": lambda: MemoryOrchestrator()},
-    },
+DFFMLOverlaysInstalled = SystemContext(
+    upstream=DFFML_OVERLAYS_INSTALLED,
+    # TODO(alice) We'll need to make sure we have code to instantiate and
+    # instance of a class if only a class is given an not an instance.
+    overlay=None,
+    orchestrator=MemoryOrchestrator(),
 )
 
 # Callee
diff --git a/entities/alice/alice/please/contribute/recommended_community_standards/alice/operations/github/issue.py b/entities/alice/alice/please/contribute/recommended_community_standards/alice/operations/github/issue.py
index 46d20c8c85..fff5d4928b 100644
--- a/entities/alice/alice/please/contribute/recommended_community_standards/alice/operations/github/issue.py
+++ b/entities/alice/alice/please/contribute/recommended_community_standards/alice/operations/github/issue.py
@@ -18,6 +18,14 @@ from ....recommended_community_standards import AliceGitRepo, AlicePleaseContrib
 from ....dffml.operations.git.contribute import AlicePleaseContributeRecommendedCommunityStandardsOverlayGit
 
 
+GitHubIssue = NewType("GitHubIssue", str)
+
+
+@dataclasses.dataclass
+class RecommendedCommunityStandardContribution:
+    path: pathlib.Path
+    issue: GitHubIssue
+
 
 class AlicePleaseContributeRecommendedCommunityStandardsOverlayGitHubIssue:
     """
@@ -39,6 +47,7 @@ class AlicePleaseContributeRecommendedCommunityStandardsOverlayGitHubIssue:
     MetaIssueTitle = NewType("MetaIssueTitle", str)
     MetaIssueBody = NewType("MetaIssueBody", str)
 
+    # TODO This should only be run if there is a need for a README
     # body: Optional['ContributingIssueBody'] = "References:\n- https://docs.github.com/articles/setting-guidelines-for-repository-contributors/",
     async def readme_issue(
         self,
@@ -79,13 +88,40 @@ class AlicePleaseContributeRecommendedCommunityStandardsOverlayGitHubIssue:
             """
         ).lstrip()
 
-    # TODO(alice) There is a bug with Optional which can be revield by use here
+
     @staticmethod
+    async def readme_contribution(
+        issue: "ReadmeIssue",
+        path: AlicePleaseContributeRecommendedCommunityStandards.ReadmePath,
+    ) -> RecommendedCommunityStandardContribution:
+        return RecommendedCommunityStandardContribution(
+            path=path,
+            issue=issue,
+        )
+
+
+    """
+    @dffml.op(
+        stage=dffml.Stage.OUTPUT,
+    )
+    async def collect_recommended_community_standard_contributions(
+        self,
+    ) -> List[RecommendedCommunityStandardContribution]:
+        async with self.octx.ictx.definitions(self.ctx) as od:
+            return [item async for item in od.inputs(RecommendedCommunityStandardContribution)]
+    """
+
+
+    # TODO(alice) There is a bug with Optional which can be revield by use here
     def meta_issue_body(
         repo: AliceGitRepo,
         base: AlicePleaseContributeRecommendedCommunityStandardsOverlayGit.BaseBranch,
-        readme_path: AlicePleaseContributeRecommendedCommunityStandards.ReadmePath,
-        readme_issue: ReadmeIssue,
+        # recommended_community_standard_contributions: List[RecommendedCommunityStandardContribution],
+        # TODO On @op inspect paramter if Collect is found on an input, wrap the
+        # operation in a subflow and add a generic version of
+        # collect_recommended_community_standard_contributions to the flow as an
+        # autostart or triggered via auto start operation.
+        # recommended_community_standard_contributions: Collect[List[RecommendedCommunityStandardContribution]],
     ) -> "MetaIssueBody":
         """
         >>> AlicePleaseContributeRecommendedCommunityStandardsGitHubIssueOverlay.meta_issue_body(
@@ -98,6 +134,7 @@ class AlicePleaseContributeRecommendedCommunityStandardsOverlayGitHubIssue:
         - [] [License](https://github.com/intel/dffml/blob/main/LICENSE)
         - [] Security
         """
+        readme_issue, readme_path = recommended_community_standard_contributions[0]
         return "\n".join(
             [
                 "- ["

johnandersen777 · 2022-07-28T10:28:59Z

johnandersen777
Jul 28, 2022
Author

2022-07-28 Engineering Logs

1 reply

johnandersen777 Jul 28, 2022
Author

2022-07-28 @pdxjohnny Engineering Logs

TODO
- Get involved in SCITT
  - Meetings
    - https://docs.google.com/document/d/1vf-EliXByhg5HZfgVbTqZhfaJFCmvMdQuZ4tC-Eq6wg/edit#
    - Weekly Monday at 8 AM Pacific
      - Joining today
    - https://armltd.zoom.us/j/99133885299?pwd=b0w4aGorRkpjL3ZHa2NPSmRiNHpXUT09
  - Mailing list
    - https://www.ietf.org/mailman/listinfo/scitt
    - https://mailarchive.ietf.org/arch/browse/scitt/
  - Slack
    - https://mailarchive.ietf.org/arch/msg/scitt/PbvoKOX996cNHJEOrjReaNlum64/
    - Going to email Orie Steele orie (at) transmute.industries to ask for an invite.
- Kick off OSS scans
  - Targeting collaboration with CRob on metrics insertion to OpenSSF DB
- Finish Q3 plans (Gantt chart, meeting templates, etc.)
  - Generate template for auto creation to fill every meeting / fillable pre-meeting
- Overlay to alice shouldi contribute to create git repos when found from forks of PyPi packages
  - Associated tutorial
    - Linked from README
- Finish out alice please contribute recommended community standards
  dynamic opimp for meta issue body creation
  - Associated tutorial
    - Linked from README and CONTRIBUTING
- Software Analysis Trinity diagram showing Human Intent, Static Analysis, and Dynamic Analysis to represent the soul of the software / entity and the process taken to improve it.
  - SoftwareAnalysisTrinity.drawio.xml
Noticed that we have an issue with adding new files and locking. The current
lock is on the git_repository/GitRepoSpec.
- We then convert to AliceGitRepo, at which point anything take AliceGitRepo
- alice: please: contribute: recommended community standards: Refactoring into overlays associated with each file contributed
  - Completed in 1a71dbe

43

(Pdb) custom_run_dataflow_ctx.config.dataflow.seed.append(dffml.Input(value=repo, definition=definition, origin=('alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:alice_contribute_readme', 'result')))
(Pdb) custom_run_dataflow_ctx.config.dataflow.seed
[Input(value=origin, definition=writable.github.remote.origin), Input(value=master, definition=repo.git.base.branch), Input(value=GitRepoSpec(directory='/tmp/dffml-feature-git-hxnacg5_', URL='https://github.com/pdxjohnny/testaaaa'), definition=ReadmeGitRepo)]

Attempting to figure out why an operation is not being called
- contribute_readme_md should be getting base, but is not.

{'_': {ReadmeGitRepo: [Input(value=GitRepoSpec(directory='/tmp/dffml-feature-git-68ghk7vd', URL='https://github.com/pdxjohnny/testaaaa'), definition=ReadmeGitRepo)],
       ReadmeIssue: [Input(value=https://github.com/pdxjohnny/testaaaa/issues/108, definition=ReadmeIssue)],
       ReadmePath: [Input(value=/tmp/dffml-feature-git-68ghk7vd/README.md, definition=ReadmePath)],
       github.pr.body: [Input(value=Closes: https://github.com/pdxjohnny/testaaaa/issues/108, definition=github.pr.body)],
       repo.git.base.branch: [Input(value=master, definition=repo.git.base.branch)],
       repo.readme.git.commit.message: [Input(value=Recommended Community Standard: README

Closes: https://github.com/pdxjohnny/testaaaa/issues/108
, definition=repo.readme.git.commit.message)],
       writable.github.remote.origin: [Input(value=origin, definition=writable.github.remote.origin)]},
 'alternate_definitions': [],
 'by_origin': {('alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGit:determin_base_branch', 'result'): [Input(value=master, definition=repo.git.base.branch)],
               ('alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGitHub:github_owns_remote', 'result'): [Input(value=origin, definition=writable.github.remote.origin)],
               ('alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:alice_contribute_readme', 'result'): [Input(value=GitRepoSpec(directory='/tmp/dffml-feature-git-68ghk7vd', URL='https://github.com/pdxjohnny/testaaaa'), definition=ReadmeGitRepo)],
               ('alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:create_readme_file_if_not_exists', 'result'): [Input(value=/tmp/dffml-feature-git-68ghk7vd/README.md, definition=ReadmePath)],
               ('alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:readme_commit_message', 'result'): [Input(value=Recommended Community Standard: README

Closes: https://github.com/pdxjohnny/testaaaa/issues/108
, definition=repo.readme.git.commit.message)],
               ('alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:readme_issue', 'result'): [Input(value=https://github.com/pdxjohnny/testaaaa/issues/108, definition=ReadmeIssue)],
               ('alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:readme_pr_body', 'result'): [Input(value=Closes: https://github.com/pdxjohnny/testaaaa/issues/108, definition=github.pr.body)]},
 'check_for_default_value': [repo.git.base.branch],
 'contexts': [MemoryInputNetworkContextEntry(ctx=Input(value=GitRepoSpec(directory='/tmp/dffml-feature-git-68ghk7vd', URL='https://github.com/pdxjohnny/testaaaa'), definition=ReadmeGitRepo), definitions={ReadmeGitRepo: [Input(value=GitRepoSpec(directory='/tmp/dffml-feature-git-68ghk7vd', URL='https://github.com/pdxjohnny/testaaaa'), definition=ReadmeGitRepo)], writable.github.remote.origin: [Input(value=origin, definition=writable.github.remote.origin)], repo.git.base.branch: [Input(value=master, definition=repo.git.base.branch)], ReadmePath: [Input(value=/tmp/dffml-feature-git-68ghk7vd/README.md, definition=ReadmePath)], ReadmeIssue: [Input(value=https://github.com/pdxjohnny/testaaaa/issues/108, definition=ReadmeIssue)], repo.readme.git.commit.message: [Input(value=Recommended Community Standard: README

Closes: https://github.com/pdxjohnny/testaaaa/issues/108
, definition=repo.readme.git.commit.message)], github.pr.body: [Input(value=Closes: https://github.com/pdxjohnny/testaaaa/issues/108, definition=github.pr.body)]}, by_origin={('alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:alice_contribute_readme', 'result'): [Input(value=GitRepoSpec(directory='/tmp/dffml-feature-git-68ghk7vd', URL='https://github.com/pdxjohnny/testaaaa'), definition=ReadmeGitRepo)], ('alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGitHub:github_owns_remote', 'result'): [Input(value=origin, definition=writable.github.remote.origin)], ('alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGit:determin_base_branch', 'result'): [Input(value=master, definition=repo.git.base.branch)], ('alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:create_readme_file_if_not_exists', 'result'): [Input(value=/tmp/dffml-feature-git-68ghk7vd/README.md, definition=ReadmePath)], ('alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:readme_issue', 'result'): [Input(value=https://github.com/pdxjohnny/testaaaa/issues/108, definition=ReadmeIssue)], ('alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:readme_commit_message', 'result'): [Input(value=Recommended Community Standard: README

Closes: https://github.com/pdxjohnny/testaaaa/issues/108
, definition=repo.readme.git.commit.message)], ('alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:readme_pr_body', 'result'): [Input(value=Closes: https://github.com/pdxjohnny/testaaaa/issues/108, definition=github.pr.body)]})],
 'ctx': Input(value=GitRepoSpec(directory='/tmp/dffml-feature-git-68ghk7vd', URL='https://github.com/pdxjohnny/testaaaa'), definition=ReadmeGitRepo),
 'dataflow': <dffml.df.types.DataFlow object at 0x7f177ec3b7f0>,
 'definition': repo.git.base.branch,
 'gather': {'base': [],
            'repo': [Input(value=GitRepoSpec(directory='/tmp/dffml-feature-git-68ghk7vd', URL='https://github.com/pdxjohnny/testaaaa'), definition=ReadmeGitRepo)]},
 'handle_string': "Input(value=GitRepoSpec(directory='/tmp/dffml-feature-git-68ghk7vd', "
                  "URL='https://github.com/pdxjohnny/testaaaa'), "
                  'definition=ReadmeGitRepo)',
 'input_flow': InputFlow(inputs={'repo': [{'alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:alice_contribute_readme': 'result'}], 'base': ['seed'], 'commit_message': [{'alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:readme_commit_message': 'result'}]}, conditions=[]),
 'input_name': 'base',
 'input_source': 'seed',
 'input_sources': ['seed'],
 'item': Input(value=GitRepoSpec(directory='/tmp/dffml-feature-git-68ghk7vd', URL='https://github.com/pdxjohnny/testaaaa'), definition=ReadmeGitRepo),
 'operation': Operation(name='alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:contribute_readme_md', inputs={'repo': ReadmeGitRepo, 'base': repo.git.base.branch, 'commit_message': repo.readme.git.commit.message}, outputs={'result': repo.readme.git.branch}, stage=<Stage.PROCESSING: 'processing'>, conditions=[], expand=[], instance_name='alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:contribute_readme_md', validator=False, retry=0),
 'origin': 'seed',
 'origins': ['seed'],
 'pprint': <function pprint at 0x7f1782d94670>,
 'rctx': <dffml.df.memory.MemoryRedundancyCheckerContext object at 0x7f177ec5e220>,
 'self': <dffml.df.memory.MemoryInputNetworkContext object at 0x7f177ec3ba30>}
> /home/pdxjohnny/Documents/python/dffml/dffml/df/memory.py(788)gather_inputs()
-> return
(Pdb) gather
{'repo': [Input(value=GitRepoSpec(directory='/tmp/dffml-feature-git-68ghk7vd', URL='https://github.com/pdxjohnny/testaaaa'), definition=ReadmeGitRepo)], 'base': []}
(Pdb) operation.inputs
{'repo': ReadmeGitRepo, 'base': repo.git.base.branch, 'commit_message': repo.readme.git.commit.message}
(Pdb) self.ctxhd.keys()
dict_keys(["Input(value=GitRepoSpec(directory='/tmp/dffml-feature-git-68ghk7vd', URL='https://github.com/pdxjohnny/testaaaa'), definition=ReadmeGitRepo)"])
(Pdb) from pprint import pprint
(Pdb) pprint(inputs.definitions)
{ReadmeGitRepo: [Input(value=GitRepoSpec(directory='/tmp/dffml-feature-git-68ghk7vd', URL='https://github.com/pdxjohnny/testaaaa'), definition=ReadmeGitRepo)],
 ReadmeIssue: [Input(value=https://github.com/pdxjohnny/testaaaa/issues/108, definition=ReadmeIssue)],
 ReadmePath: [Input(value=/tmp/dffml-feature-git-68ghk7vd/README.md, definition=ReadmePath)],
 github.pr.body: [Input(value=Closes: https://github.com/pdxjohnny/testaaaa/issues/108, definition=github.pr.body)],
 repo.git.base.branch: [Input(value=master, definition=repo.git.base.branch)],
 repo.readme.git.commit.message: [Input(value=Recommended Community Standard: README

Closes: https://github.com/pdxjohnny/testaaaa/issues/108
, definition=repo.readme.git.commit.message)],
 writable.github.remote.origin: [Input(value=origin, definition=writable.github.remote.origin)]}
(Pdb) gather
{'repo': [Input(value=GitRepoSpec(directory='/tmp/dffml-feature-git-68ghk7vd', URL='https://github.com/pdxjohnny/testaaaa'), definition=ReadmeGitRepo)], 'base': []}
(Pdb) operation.inputs
{'repo': ReadmeGitRepo, 'base': repo.git.base.branch, 'commit_message': repo.readme.git.commit.message}

Suspect discarded because of mismatched origin, if not that, will check definition
- Found out that it was seed vs. output origin mismatch
- Found out that BaseBranch comes from OverlayGit
- Registered OverlayGit as an overlay of OverlayReadme to that it's definitions get loaded
  - This way auto_flow will make the expected origin the output from OverlayGit operations
    rather than seed (the default when no matching outputs are seen on DataFlow init).
- We found it created an infinite loop
  - Will try reusing redundancy checker, that seems to be doing well
alice: please: contribute: recommended community standards: github: GraphQL: was submitted too quickly #1408
Now debugging why readme_pr not called, OverlayGit definitions were seen earlier
on subflow start to be present, must be something else.
- The logs tell us that alice_contribute_readme is returning None, which means
  that the downstream operation is not called, since None means no return value
  in this case.

DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:alice_contribute_readme Outputs: None

Future
- run_custom Optionally support forward subflow
TODO
- Set definition proprety AliceGitRepo.lock to True

44

Found out that util: subprocess: run command events: Do not return after yield of stdout/err
- Fixed in b6eea6e

$ (for i in $(echo determin_base_branch readme_pr_body contribute_readme_md github_owns_remote alice_contribute_readme); do grep -rn "${i} Outputs" .output/2022-07-28-14-11.txt; done) | sort | uniq | sort
354:DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGitHub:github_owns_remote Outputs: {'result': 'origin'}
361:DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGit:determin_base_branch Outputs: {'result': 'master'}
450:DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:readme_pr_body Outputs: {'result': 'Closes: https://github.com/pdxjohnny/testaaaa/issues/188'}
472:DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:contribute_readme_md Outputs: {'result': 'alice-contribute-recommended-community-standards-readme'}
479:DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:alice_contribute_readme Outputs: None

(Pdb)
DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:contribute_readme_md Outputs: {'result': 'alice-contribute-recommended-community-standards-readme'}
(Pdb) pprint(readme_dataflow.flow['alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:readme_pr'].inputs)
{'base': [{'alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGit:determin_base_branch': 'result'}],
 'body': [{'alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:readme_pr_body': 'result'}],
 'head': [{'alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:contribute_readme_md': 'result'}],
 'origin': ['seed'],
 'repo': [{'alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:alice_contribute_readme': 'result'}],
 'title': [{'alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:readme_pr_title': 'result'}]}

origin is set to seed
- 'origin': ['seed'] was there because OverlayGitHub.github_owns_remote is not in the flow
- We forgot add it to entry_points.txt, added

$ dffml service dev export alice.cli:AlicePleaseContributeCLIDataFlow | tee alice.please.contribute.recommended_community_standards.json
$ (echo -e 'HTTP/1.0 200 OK\n' && dffml dataflow diagram -shortname alice.please.contribute.recommended_community_standards.json) | nc -Nlp 9999;

Opens
- guessed_repo_string_means_no_git_branch_given is feeding git_repo_default_branch but dffml dataflow diagram just have a bug because it's not showing the connection.

graph TD
subgraph a759a07029077edc5c37fea0326fa281[Processing Stage]
style a759a07029077edc5c37fea0326fa281 fill:#afd388b5,stroke:#a4ca7a
subgraph 8cfb8cd5b8620de4a7ebe0dfec00771a[cli_has_repos]
style 8cfb8cd5b8620de4a7ebe0dfec00771a fill:#fff4de,stroke:#cece71
d493c90433d19f11f33c2d72cd144940[cli_has_repos]
e07552ee3b6b7696cb3ddd786222eaad(cmd)
e07552ee3b6b7696cb3ddd786222eaad --> d493c90433d19f11f33c2d72cd144940
cee6b5fdd0b6fbd0539cdcdc7f5a3324(wanted)
cee6b5fdd0b6fbd0539cdcdc7f5a3324 --> d493c90433d19f11f33c2d72cd144940
79e1ea6822bff603a835fb8ee80c7ff3(result)
d493c90433d19f11f33c2d72cd144940 --> 79e1ea6822bff603a835fb8ee80c7ff3
end
subgraph 0c2b64320fb5666a034794bb2195ecf0[cli_is_asking_for_recommended_community_standards]
style 0c2b64320fb5666a034794bb2195ecf0 fill:#fff4de,stroke:#cece71
222ee6c0209f1f1b7a782bc1276868c7[cli_is_asking_for_recommended_community_standards]
330f463830aa97e88917d5a9d1c21500(cmd)
330f463830aa97e88917d5a9d1c21500 --> 222ee6c0209f1f1b7a782bc1276868c7
ba29b52e9c5aa88ea1caeeff29bfd491(result)
222ee6c0209f1f1b7a782bc1276868c7 --> ba29b52e9c5aa88ea1caeeff29bfd491
end
subgraph eac58e8db2b55cb9cc5474aaa402c93e[cli_is_meant_on_this_repo]
style eac58e8db2b55cb9cc5474aaa402c93e fill:#fff4de,stroke:#cece71
6c819ad0228b0e7094b33e0634da9a38[cli_is_meant_on_this_repo]
dc7c5f0836f7d2564c402bf956722672(cmd)
dc7c5f0836f7d2564c402bf956722672 --> 6c819ad0228b0e7094b33e0634da9a38
58d8518cb0d6ef6ad35dc242486f1beb(wanted)
58d8518cb0d6ef6ad35dc242486f1beb --> 6c819ad0228b0e7094b33e0634da9a38
135ee61e3402d6fcbd7a219b0b4ccd73(result)
6c819ad0228b0e7094b33e0634da9a38 --> 135ee61e3402d6fcbd7a219b0b4ccd73
end
subgraph 37887bf260c5c8e9bd18038401008bbc[cli_run_on_repo]
style 37887bf260c5c8e9bd18038401008bbc fill:#fff4de,stroke:#cece71
9d1042f33352800e54d98c9c5a4223df[cli_run_on_repo]
e824ae072860bc545fc7d55aa0bca479(repo)
e824ae072860bc545fc7d55aa0bca479 --> 9d1042f33352800e54d98c9c5a4223df
40109d487bb9f08608d8c5f6e747042f(result)
9d1042f33352800e54d98c9c5a4223df --> 40109d487bb9f08608d8c5f6e747042f
end
subgraph 66ecd0c1f2e08941c443ec9cd89ec589[guess_repo_string_is_directory]
style 66ecd0c1f2e08941c443ec9cd89ec589 fill:#fff4de,stroke:#cece71
737d719a0c348ff65456024ddbc530fe[guess_repo_string_is_directory]
33d806f9b732bfd6b96ae2e9e4243a68(repo_string)
33d806f9b732bfd6b96ae2e9e4243a68 --> 737d719a0c348ff65456024ddbc530fe
dd5aab190ce844673819298c5b8fde76(result)
737d719a0c348ff65456024ddbc530fe --> dd5aab190ce844673819298c5b8fde76
end
subgraph 4ea6696419c4a0862a4f63ea1f60c751[create_branch_if_none_exists]
style 4ea6696419c4a0862a4f63ea1f60c751 fill:#fff4de,stroke:#cece71
502369b37882b300d6620d5b4020f5b2[create_branch_if_none_exists]
fdcb9b6113856222e30e093f7c38065e(name)
fdcb9b6113856222e30e093f7c38065e --> 502369b37882b300d6620d5b4020f5b2
bdcf4b078985f4a390e4ed4beacffa65(repo)
bdcf4b078985f4a390e4ed4beacffa65 --> 502369b37882b300d6620d5b4020f5b2
5a5493ab86ab4053f1d44302e7bdddd6(result)
502369b37882b300d6620d5b4020f5b2 --> 5a5493ab86ab4053f1d44302e7bdddd6
end
subgraph b1d510183f6a4c3fde207a4656c72cb4[determin_base_branch]
style b1d510183f6a4c3fde207a4656c72cb4 fill:#fff4de,stroke:#cece71
476aecd4d4d712cda1879feba46ea109[determin_base_branch]
ff47cf65b58262acec28507f4427de45(default_branch)
ff47cf65b58262acec28507f4427de45 --> 476aecd4d4d712cda1879feba46ea109
150204cd2d5a921deb53c312418379a1(result)
476aecd4d4d712cda1879feba46ea109 --> 150204cd2d5a921deb53c312418379a1
end
subgraph 2a08ff341f159c170b7fe017eaad2f18[git_repo_to_alice_git_repo]
style 2a08ff341f159c170b7fe017eaad2f18 fill:#fff4de,stroke:#cece71
7f74112f6d30c6289caa0a000e87edab[git_repo_to_alice_git_repo]
e58180baf478fe910359358a3fa02234(repo)
e58180baf478fe910359358a3fa02234 --> 7f74112f6d30c6289caa0a000e87edab
9b92d5a346885079a2821c4d27cb5174(result)
7f74112f6d30c6289caa0a000e87edab --> 9b92d5a346885079a2821c4d27cb5174
end
subgraph b5d35aa8a8dcd28d22d47caad02676b0[guess_repo_string_is_url]
style b5d35aa8a8dcd28d22d47caad02676b0 fill:#fff4de,stroke:#cece71
0de074e71a32e30889b8bb400cf8db9f[guess_repo_string_is_url]
c3bfe79b396a98ce2d9bfe772c9c20af(repo_string)
c3bfe79b396a98ce2d9bfe772c9c20af --> 0de074e71a32e30889b8bb400cf8db9f
2a1c620b0d510c3d8ed35deda41851c5(result)
0de074e71a32e30889b8bb400cf8db9f --> 2a1c620b0d510c3d8ed35deda41851c5
end
subgraph 60791520c6d124c0bf15e599132b0caf[guessed_repo_string_is_operations_git_url]
style 60791520c6d124c0bf15e599132b0caf fill:#fff4de,stroke:#cece71
102f173505d7b546236cdeff191369d4[guessed_repo_string_is_operations_git_url]
4934c6211334318c63a5e91530171c9b(repo_url)
4934c6211334318c63a5e91530171c9b --> 102f173505d7b546236cdeff191369d4
8d0adc31da1a0919724baf73d047743c(result)
102f173505d7b546236cdeff191369d4 --> 8d0adc31da1a0919724baf73d047743c
end
subgraph f2c7b93622447999daab403713239ada[guessed_repo_string_means_no_git_branch_given]
style f2c7b93622447999daab403713239ada fill:#fff4de,stroke:#cece71
c8294a87e7aae8f7f9cb7f53e054fed5[guessed_repo_string_means_no_git_branch_given]
5567dd8a6d7ae4fe86252db32e189a4d(repo_url)
5567dd8a6d7ae4fe86252db32e189a4d --> c8294a87e7aae8f7f9cb7f53e054fed5
d888e6b64b5e3496056088f14dab9894(result)
c8294a87e7aae8f7f9cb7f53e054fed5 --> d888e6b64b5e3496056088f14dab9894
end
subgraph 113addf4beee5305fdc79d2363608f9d[github_owns_remote]
style 113addf4beee5305fdc79d2363608f9d fill:#fff4de,stroke:#cece71
049b72b81b976fbb43607bfeeb0464c5[github_owns_remote]
6c2b36393ffff6be0b4ad333df2d9419(remote)
6c2b36393ffff6be0b4ad333df2d9419 --> 049b72b81b976fbb43607bfeeb0464c5
19a9ee483c1743e6ecf0a2dc3b6f8c7a(repo)
19a9ee483c1743e6ecf0a2dc3b6f8c7a --> 049b72b81b976fbb43607bfeeb0464c5
b4cff8d194413f436d94f9d84ece0262(result)
049b72b81b976fbb43607bfeeb0464c5 --> b4cff8d194413f436d94f9d84ece0262
end
subgraph 43a22312a3d4f5c995c54c5196acc50a[create_meta_issue]
style 43a22312a3d4f5c995c54c5196acc50a fill:#fff4de,stroke:#cece71
d2345f23e5ef9f54c591c4a687c24575[create_meta_issue]
1d79010ee1550f057c531130814c40b9(body)
1d79010ee1550f057c531130814c40b9 --> d2345f23e5ef9f54c591c4a687c24575
712d4318e59bd2dc629f0ddebb257ca3(repo)
712d4318e59bd2dc629f0ddebb257ca3 --> d2345f23e5ef9f54c591c4a687c24575
38a94f1c2162803f571489d707d61021(title)
38a94f1c2162803f571489d707d61021 --> d2345f23e5ef9f54c591c4a687c24575
2b22b4998ac3e6a64d82e0147e71ee1b(result)
d2345f23e5ef9f54c591c4a687c24575 --> 2b22b4998ac3e6a64d82e0147e71ee1b
end
subgraph f77af509c413b86b6cd7e107cc623c73[meta_issue_body]
style f77af509c413b86b6cd7e107cc623c73 fill:#fff4de,stroke:#cece71
69a9852570720a3d35cb9dd52a281f71[meta_issue_body]
480d1cc478d23858e92d61225349b674(base)
480d1cc478d23858e92d61225349b674 --> 69a9852570720a3d35cb9dd52a281f71
37035ea5a06a282bdc1e1de24090a36d(readme_issue)
37035ea5a06a282bdc1e1de24090a36d --> 69a9852570720a3d35cb9dd52a281f71
fdf0dbb8ca47ee9022b3daeb8c7df9c0(readme_path)
fdf0dbb8ca47ee9022b3daeb8c7df9c0 --> 69a9852570720a3d35cb9dd52a281f71
428ca84f627c695362652cc7531fc27b(repo)
428ca84f627c695362652cc7531fc27b --> 69a9852570720a3d35cb9dd52a281f71
0cd9eb1ffb3c56d2b0a4359f800b1f20(result)
69a9852570720a3d35cb9dd52a281f71 --> 0cd9eb1ffb3c56d2b0a4359f800b1f20
end
subgraph 8506cba6514466fb6d65f33ace4b0eac[alice_contribute_readme]
style 8506cba6514466fb6d65f33ace4b0eac fill:#fff4de,stroke:#cece71
d4507d3d1c3fbf3e7e373eae24797667[alice_contribute_readme]
68cf7d6869d027ca46a5fb4dbf7001d1(repo)
68cf7d6869d027ca46a5fb4dbf7001d1 --> d4507d3d1c3fbf3e7e373eae24797667
2f9316539862f119f7c525bf9061e974(result)
d4507d3d1c3fbf3e7e373eae24797667 --> 2f9316539862f119f7c525bf9061e974
end
subgraph 4233e6dc67cba131d4ef005af9c02959[contribute_readme_md]
style 4233e6dc67cba131d4ef005af9c02959 fill:#fff4de,stroke:#cece71
3db0ee5d6ab83886bded5afd86f3f88f[contribute_readme_md]
37044e4d8610abe13849bc71a5cb7591(base)
37044e4d8610abe13849bc71a5cb7591 --> 3db0ee5d6ab83886bded5afd86f3f88f
631c051fe6050ae8f8fc3321ed00802d(commit_message)
631c051fe6050ae8f8fc3321ed00802d --> 3db0ee5d6ab83886bded5afd86f3f88f
182194bab776fc9bc406ed573d621b68(repo)
182194bab776fc9bc406ed573d621b68 --> 3db0ee5d6ab83886bded5afd86f3f88f
0ee9f524d2db12be854fe611fa8126dd(result)
3db0ee5d6ab83886bded5afd86f3f88f --> 0ee9f524d2db12be854fe611fa8126dd
end
subgraph a6080d9c45eb5f806a47152a18bf7830[create_readme_file_if_not_exists]
style a6080d9c45eb5f806a47152a18bf7830 fill:#fff4de,stroke:#cece71
67e388f508dd96084c37d236a2c67e67[create_readme_file_if_not_exists]
54faf20bfdca0e63d07efb3e5a984cf1(readme_contents)
54faf20bfdca0e63d07efb3e5a984cf1 --> 67e388f508dd96084c37d236a2c67e67
8c089c362960ccf181742334a3dccaea(repo)
8c089c362960ccf181742334a3dccaea --> 67e388f508dd96084c37d236a2c67e67
5cc65e17d40e6a7223c1504f1c4b0d2a(result)
67e388f508dd96084c37d236a2c67e67 --> 5cc65e17d40e6a7223c1504f1c4b0d2a
end
subgraph e7757158127e9845b2915c16a7fa80c5[readme_commit_message]
style e7757158127e9845b2915c16a7fa80c5 fill:#fff4de,stroke:#cece71
562bdc535c7cebfc66dba920b1a17540[readme_commit_message]
0af5cbea9050874a0a3cba73bb61f892(issue_url)
0af5cbea9050874a0a3cba73bb61f892 --> 562bdc535c7cebfc66dba920b1a17540
2641f3b67327fb7518ee34a3a40b0755(result)
562bdc535c7cebfc66dba920b1a17540 --> 2641f3b67327fb7518ee34a3a40b0755
end
subgraph cf99ff6fad80e9c21266b43fd67b2f7b[readme_issue]
style cf99ff6fad80e9c21266b43fd67b2f7b fill:#fff4de,stroke:#cece71
da44417f891a945085590baafffc2bdb[readme_issue]
d519830ab4e07ec391038e8581889ac3(body)
d519830ab4e07ec391038e8581889ac3 --> da44417f891a945085590baafffc2bdb
268852aa3fa8ab0864a32abae5a333f7(repo)
268852aa3fa8ab0864a32abae5a333f7 --> da44417f891a945085590baafffc2bdb
77a11dd29af309cf43ed321446c4bf01(title)
77a11dd29af309cf43ed321446c4bf01 --> da44417f891a945085590baafffc2bdb
1d2360c9da18fac0b6ec142df8f3fbda(result)
da44417f891a945085590baafffc2bdb --> 1d2360c9da18fac0b6ec142df8f3fbda
end
subgraph 7ec0442cf2d95c367912e8abee09b217[readme_pr]
style 7ec0442cf2d95c367912e8abee09b217 fill:#fff4de,stroke:#cece71
bb314dc452cde5b6af5ea94dd277ba40[readme_pr]
127d77c3047facc1daa621148c5a0a1d(base)
127d77c3047facc1daa621148c5a0a1d --> bb314dc452cde5b6af5ea94dd277ba40
cb421e4de153cbb912f7fbe57e4ad734(body)
cb421e4de153cbb912f7fbe57e4ad734 --> bb314dc452cde5b6af5ea94dd277ba40
cbf7a0b88c0a41953b245303f3e9a0d3(head)
cbf7a0b88c0a41953b245303f3e9a0d3 --> bb314dc452cde5b6af5ea94dd277ba40
e5f9ad44448abd2469b3fd9831f3d159(origin)
e5f9ad44448abd2469b3fd9831f3d159 --> bb314dc452cde5b6af5ea94dd277ba40
a35aee6711d240378eb57a3932537ca1(repo)
a35aee6711d240378eb57a3932537ca1 --> bb314dc452cde5b6af5ea94dd277ba40
dfcce88a7d605d46bf17de1159fbe5ad(title)
dfcce88a7d605d46bf17de1159fbe5ad --> bb314dc452cde5b6af5ea94dd277ba40
a210a7890a7bea8d629368e02da3d806(result)
bb314dc452cde5b6af5ea94dd277ba40 --> a210a7890a7bea8d629368e02da3d806
end
subgraph 227eabb1f1c5cc0bc931714a03049e27[readme_pr_body]
style 227eabb1f1c5cc0bc931714a03049e27 fill:#fff4de,stroke:#cece71
2aea976396cfe68dacd9bc7d4a3f0cba[readme_pr_body]
c5dfd309617c909b852afe0b4ae4a178(readme_issue)
c5dfd309617c909b852afe0b4ae4a178 --> 2aea976396cfe68dacd9bc7d4a3f0cba
40ddb5b508cb5643e7c91f7abdb72b84(result)
2aea976396cfe68dacd9bc7d4a3f0cba --> 40ddb5b508cb5643e7c91f7abdb72b84
end
subgraph 48687c84e69b3db0acca625cbe2e6b49[readme_pr_title]
style 48687c84e69b3db0acca625cbe2e6b49 fill:#fff4de,stroke:#cece71
d8668ff93f41bc241c8c540199cd7453[readme_pr_title]
3b2137dd1c61d0dac7d4e40fd6746cfb(readme_issue)
3b2137dd1c61d0dac7d4e40fd6746cfb --> d8668ff93f41bc241c8c540199cd7453
956e024fde513b3a449eac9ee42d6ab3(result)
d8668ff93f41bc241c8c540199cd7453 --> 956e024fde513b3a449eac9ee42d6ab3
end
subgraph d3ec0ac85209a7256c89d20f758f09f4[check_if_valid_git_repository_URL]
style d3ec0ac85209a7256c89d20f758f09f4 fill:#fff4de,stroke:#cece71
f577c71443f6b04596b3fe0511326c40[check_if_valid_git_repository_URL]
7440e73a8e8f864097f42162b74f2762(URL)
7440e73a8e8f864097f42162b74f2762 --> f577c71443f6b04596b3fe0511326c40
8e39b501b41c5d0e4596318f80a03210(valid)
f577c71443f6b04596b3fe0511326c40 --> 8e39b501b41c5d0e4596318f80a03210
end
subgraph af8da22d1318d911f29b95e687f87c5d[clone_git_repo]
style af8da22d1318d911f29b95e687f87c5d fill:#fff4de,stroke:#cece71
155b8fdb5524f6bfd5adbae4940ad8d5[clone_git_repo]
eed77b9eea541e0c378c67395351099c(URL)
eed77b9eea541e0c378c67395351099c --> 155b8fdb5524f6bfd5adbae4940ad8d5
8b5928cd265dd2c44d67d076f60c8b05(ssh_key)
8b5928cd265dd2c44d67d076f60c8b05 --> 155b8fdb5524f6bfd5adbae4940ad8d5
4e1d5ea96e050e46ebf95ebc0713d54c(repo)
155b8fdb5524f6bfd5adbae4940ad8d5 --> 4e1d5ea96e050e46ebf95ebc0713d54c
6a44de06a4a3518b939b27c790f6cdce{valid_git_repository_URL}
6a44de06a4a3518b939b27c790f6cdce --> 155b8fdb5524f6bfd5adbae4940ad8d5
end
subgraph d3d91578caf34c0ae944b17853783406[git_repo_default_branch]
style d3d91578caf34c0ae944b17853783406 fill:#fff4de,stroke:#cece71
546062a96122df465d2631f31df4e9e3[git_repo_default_branch]
181f1b33df4d795fbad2911ec7087e86(repo)
181f1b33df4d795fbad2911ec7087e86 --> 546062a96122df465d2631f31df4e9e3
57651c1bcd24b794dfc8d1794ab556d5(branch)
546062a96122df465d2631f31df4e9e3 --> 57651c1bcd24b794dfc8d1794ab556d5
5ed1ab77e726d7efdcc41e9e2f8039c6(remote)
546062a96122df465d2631f31df4e9e3 --> 5ed1ab77e726d7efdcc41e9e2f8039c6
4c3cdd5f15b7a846d291aac089e8a622{no_git_branch_given}
4c3cdd5f15b7a846d291aac089e8a622 --> 546062a96122df465d2631f31df4e9e3
end
end
subgraph a4827add25f5c7d5895c5728b74e2beb[Cleanup Stage]
style a4827add25f5c7d5895c5728b74e2beb fill:#afd388b5,stroke:#a4ca7a
end
subgraph 58ca4d24d2767176f196436c2890b926[Output Stage]
style 58ca4d24d2767176f196436c2890b926 fill:#afd388b5,stroke:#a4ca7a
end
subgraph inputs[Inputs]
style inputs fill:#f6dbf9,stroke:#a178ca
128516cfa09b0383023eab52ee24878a(seed<br>dffml.util.cli.CMD)
128516cfa09b0383023eab52ee24878a --> e07552ee3b6b7696cb3ddd786222eaad
ba29b52e9c5aa88ea1caeeff29bfd491 --> cee6b5fdd0b6fbd0539cdcdc7f5a3324
128516cfa09b0383023eab52ee24878a(seed<br>dffml.util.cli.CMD)
128516cfa09b0383023eab52ee24878a --> 330f463830aa97e88917d5a9d1c21500
128516cfa09b0383023eab52ee24878a(seed<br>dffml.util.cli.CMD)
128516cfa09b0383023eab52ee24878a --> dc7c5f0836f7d2564c402bf956722672
ba29b52e9c5aa88ea1caeeff29bfd491 --> 58d8518cb0d6ef6ad35dc242486f1beb
79e1ea6822bff603a835fb8ee80c7ff3 --> e824ae072860bc545fc7d55aa0bca479
135ee61e3402d6fcbd7a219b0b4ccd73 --> e824ae072860bc545fc7d55aa0bca479
40109d487bb9f08608d8c5f6e747042f --> 33d806f9b732bfd6b96ae2e9e4243a68
21ccfd2c550bd853d28581f0b0c9f9fe(seed<br>default.branch.name)
21ccfd2c550bd853d28581f0b0c9f9fe --> fdcb9b6113856222e30e093f7c38065e
dd5aab190ce844673819298c5b8fde76 --> bdcf4b078985f4a390e4ed4beacffa65
9b92d5a346885079a2821c4d27cb5174 --> bdcf4b078985f4a390e4ed4beacffa65
5a5493ab86ab4053f1d44302e7bdddd6 --> ff47cf65b58262acec28507f4427de45
57651c1bcd24b794dfc8d1794ab556d5 --> ff47cf65b58262acec28507f4427de45
4e1d5ea96e050e46ebf95ebc0713d54c --> e58180baf478fe910359358a3fa02234
40109d487bb9f08608d8c5f6e747042f --> c3bfe79b396a98ce2d9bfe772c9c20af
2a1c620b0d510c3d8ed35deda41851c5 --> 4934c6211334318c63a5e91530171c9b
2a1c620b0d510c3d8ed35deda41851c5 --> 5567dd8a6d7ae4fe86252db32e189a4d
5ed1ab77e726d7efdcc41e9e2f8039c6 --> 6c2b36393ffff6be0b4ad333df2d9419
dd5aab190ce844673819298c5b8fde76 --> 19a9ee483c1743e6ecf0a2dc3b6f8c7a
9b92d5a346885079a2821c4d27cb5174 --> 19a9ee483c1743e6ecf0a2dc3b6f8c7a
0cd9eb1ffb3c56d2b0a4359f800b1f20 --> 1d79010ee1550f057c531130814c40b9
dd5aab190ce844673819298c5b8fde76 --> 712d4318e59bd2dc629f0ddebb257ca3
9b92d5a346885079a2821c4d27cb5174 --> 712d4318e59bd2dc629f0ddebb257ca3
e7ad3469d98c3bd160363dbc47e2d741(seed<br>MetaIssueTitle)
e7ad3469d98c3bd160363dbc47e2d741 --> 38a94f1c2162803f571489d707d61021
150204cd2d5a921deb53c312418379a1 --> 480d1cc478d23858e92d61225349b674
1d2360c9da18fac0b6ec142df8f3fbda --> 37035ea5a06a282bdc1e1de24090a36d
5cc65e17d40e6a7223c1504f1c4b0d2a --> fdf0dbb8ca47ee9022b3daeb8c7df9c0
dd5aab190ce844673819298c5b8fde76 --> 428ca84f627c695362652cc7531fc27b
9b92d5a346885079a2821c4d27cb5174 --> 428ca84f627c695362652cc7531fc27b
dd5aab190ce844673819298c5b8fde76 --> 68cf7d6869d027ca46a5fb4dbf7001d1
9b92d5a346885079a2821c4d27cb5174 --> 68cf7d6869d027ca46a5fb4dbf7001d1
150204cd2d5a921deb53c312418379a1 --> 37044e4d8610abe13849bc71a5cb7591
2641f3b67327fb7518ee34a3a40b0755 --> 631c051fe6050ae8f8fc3321ed00802d
2f9316539862f119f7c525bf9061e974 --> 182194bab776fc9bc406ed573d621b68
d2708225c1f4c95d613a2645a17a5bc0(seed<br>repo.directory.readme.contents)
d2708225c1f4c95d613a2645a17a5bc0 --> 54faf20bfdca0e63d07efb3e5a984cf1
2f9316539862f119f7c525bf9061e974 --> 8c089c362960ccf181742334a3dccaea
1d2360c9da18fac0b6ec142df8f3fbda --> 0af5cbea9050874a0a3cba73bb61f892
1daacccd02f8117e67ad3cb8686a732c(seed<br>ReadmeIssueBody)
1daacccd02f8117e67ad3cb8686a732c --> d519830ab4e07ec391038e8581889ac3
2f9316539862f119f7c525bf9061e974 --> 268852aa3fa8ab0864a32abae5a333f7
0c1ab2d4bda10e1083557833ae5c5da4(seed<br>ReadmeIssueTitle)
0c1ab2d4bda10e1083557833ae5c5da4 --> 77a11dd29af309cf43ed321446c4bf01
150204cd2d5a921deb53c312418379a1 --> 127d77c3047facc1daa621148c5a0a1d
40ddb5b508cb5643e7c91f7abdb72b84 --> cb421e4de153cbb912f7fbe57e4ad734
0ee9f524d2db12be854fe611fa8126dd --> cbf7a0b88c0a41953b245303f3e9a0d3
b4cff8d194413f436d94f9d84ece0262 --> e5f9ad44448abd2469b3fd9831f3d159
2f9316539862f119f7c525bf9061e974 --> a35aee6711d240378eb57a3932537ca1
956e024fde513b3a449eac9ee42d6ab3 --> dfcce88a7d605d46bf17de1159fbe5ad
1d2360c9da18fac0b6ec142df8f3fbda --> c5dfd309617c909b852afe0b4ae4a178
1d2360c9da18fac0b6ec142df8f3fbda --> 3b2137dd1c61d0dac7d4e40fd6746cfb
8d0adc31da1a0919724baf73d047743c --> 7440e73a8e8f864097f42162b74f2762
8d0adc31da1a0919724baf73d047743c --> eed77b9eea541e0c378c67395351099c
a6ed501edbf561fda49a0a0a3ca310f0(seed<br>git_repo_ssh_key)
a6ed501edbf561fda49a0a0a3ca310f0 --> 8b5928cd265dd2c44d67d076f60c8b05
8e39b501b41c5d0e4596318f80a03210 --> 6a44de06a4a3518b939b27c790f6cdce
4e1d5ea96e050e46ebf95ebc0713d54c --> 181f1b33df4d795fbad2911ec7087e86
end

As of f8619a6 it works without meta issue
creation: Recommended Community Standard: README johnandersen777/testaaaa#193

45

$ gif-for-cli --rows $(tput lines) --cols $(tput cols) --export=/mnt/c/Users/Johnny/Downloads/alice-search-alices-adventures-in-wonderland-1.gif "Alice's Adventures in Wonderland"

$ watch -n 0.2 'grep FEEDFACE .output/$(ls .output/ | tail -n 1) | sed -e "s/alice.please.contribute.recommended_community_standards.recommended_community_standards.//g" | grep -i repo'

johnandersen777 · 2022-07-29T12:07:56Z

johnandersen777
Jul 29, 2022
Author

2022-07-29 Engineering Logs

Alice PR: 2nd Party #1401
John's last day before sabbatical
- He will be in town but offline until 2022-08-29
Rolling Alice: 2022 Progress Reports: July Activities Recap: https://youtu.be/JDh2DARl8os
Alice is ready for contribution
- https://github.com/intel/dffml/blob/alice/docs/tutorials/rolling_alice/0001_coach_alice/0002_our_open_source_guide.md
- https://github.com/intel/dffml/blob/alice/entities/alice/CONTRIBUTING.rst
- Self fulfilling prophecy again! We can even automate our contributions to her even if we wanted to! She will eventually! :P
IETF
- Joined SCITT WG, will rejoin in September, others please do as well!
OpenSSF
- Aligned with Identifying Security Threats WG on SCITT looking like a solid direction to cross with Stream 8 for 1-2 year timeframe as Web5 space matures.
Graphics to help people get involved
- https://drive.google.com/drive/folders/1E8tZT15DNjd13jVR6xqsblgLvwTZZo_f

1 reply

johnandersen777 Jul 29, 2022
Author

2022-07-29 @pdxjohnny Engineering Logs

AppSec PNW 2022 Talk playlist: https://youtube.com/playlist?list=PLfoJYLR9vr_IAd1vYWdKCOO4YYpGFVv99
- John^2: Living Threat Models are Better Than Dead Threat Models
  - Not yet uploaded but has Alice's first live demo
https://towardsdatascience.com/installing-multiple-alternative-versions-of-python-on-ubuntu-20-04-237be5177474
- $ sudo update-alternatives --install /usr/bin/python python /usr/bin/python3 40
References
- https://tenor.com/search/alice-gifs
- https://tenor.com/view/why-thank-you-thanks-bow-thank-you-alice-in-wonderland-gif-3553903
  - Alice curtsy
- https://tenor.com/view/alice-in-wonderland-gif-26127117
  - Alice blows out unbirthday cake candle

$ alice; sleep 3; gif-for-cli -l 0 --rows $(tput lines) --cols $(tput cols) 3553903

$ gif-for-cli --rows `tput lines` --cols `tput cols` --export=alice-search-alices-adventures-in-wonderland-1.gif "Alice curtsy"
(why-thank-you-thanks-bow-thank-you-alice-in-wonderland-gif-3553903)
$ gif-for-cli --rows `tput lines` --cols `tput cols` --export=ascii-gif-alice-unbirthday-blow-out-candles-0.gif 26127117
$ gif-for-cli --rows `tput lines` --cols `tput cols` ascii-gif-alice-unbirthday-blow-out-candles-0.gif
$ echo gif-for-cli --rows `tput lines` --cols `tput cols`
gif-for-cli --rows 97 --cols 320
$ gif-for-cli -l 0 --rows `tput lines` --cols `tput cols` /mnt/c/Users/Johnny/Downloads/ascii-alices-adventures-in-wonderland-1.gif`

Exploring a Helper Around Run DataFlow run_custom

Realized we already have the lock because it's on git_repository at flow_depth=1

diff --git a/dffml/df/base.py b/dffml/df/base.py
index 4f84c1c7c8..2da0512602 100644
--- a/dffml/df/base.py
+++ b/dffml/df/base.py
@@ -404,14 +404,19 @@ def op(
                     )
 
                 definition_name = ".".join(name_list)
+                print("FEEDFACE", name, definition_name)
                 if hasattr(param_annotation, "__supertype__") and hasattr(
                     param_annotation, "__name__"
                 ):
+                    if "repo" in definition_name:
+                        breakpoint()
                     definition_name = param_annotation.__name__
+                    print("FEEDFACE", name, definition_name)
                 if inspect.isclass(param_annotation) and hasattr(
                     param_annotation, "__qualname__"
                 ):
                     definition_name = param_annotation.__qualname__
+                    print("FEEDFACE", name, definition_name)
 
                 if isinstance(param_annotation, Definition):
                     kwargs["inputs"][name] = param_annotation
diff --git a/dffml/df/types.py b/dffml/df/types.py
index f09a8a3cea..54840f58c0 100644
--- a/dffml/df/types.py
+++ b/dffml/df/types.py
@@ -44,6 +44,7 @@ APPLY_INSTALLED_OVERLAYS = _APPLY_INSTALLED_OVERLAYS()
 
 
 Expand = Union
+LockReadWrite = Union
 
 
 primitive_types = (int, float, str, bool, dict, list, bytes)
@@ -65,7 +66,7 @@ def find_primitive(new_type: Type) -> Type:
     )
 
 
-def new_type_to_defininition(new_type: Type) -> Type:
+def new_type_to_defininition(new_type: Type, lock: bool = False) -> Type:
     """
     >>> from typing import NewType
     >>> from dffml import new_type_to_defininition
@@ -77,6 +78,7 @@ def new_type_to_defininition(new_type: Type) -> Type:
     return Definition(
         name=new_type.__name__,
         primitive=find_primitive(new_type).__qualname__,
+        lock=lock,
         links=(
             create_definition(
                 find_primitive(new_type).__qualname__, new_type.__supertype__
@@ -95,7 +97,28 @@ class CouldNotDeterminePrimitive(Exception):
     """
 
 
-def resolve_if_forward_ref(param_annotation, forward_refs_from_cls):
+DEFAULT_DEFINTION_ANNOTATIONS_HANDLERS = {
+    LockReadWrite: lambda definition: setattr(definition, "lock", True),
+}
+
+
+def resolve_if_forward_ref(
+    param_annotation,
+    forward_refs_from_cls,
+    *,
+    defintion_annotations_handlers=None,
+) -> Tuple[Union["Definition", Any], bool]:
+    """
+    Return values:
+
+        param_or_definition: Union[Definition, Any]
+        lock: bool
+
+            If the definition should be locked or not.
+    """
+    if defintion_annotations_handlers is None:
+        defintion_annotations_handlers = DEFAULT_DEFINTION_ANNOTATIONS_HANDLERS
+    annotations = {}
     if isinstance(param_annotation, ForwardRef):
         param_annotation = param_annotation.__forward_arg__
     if (
@@ -104,11 +127,22 @@ def resolve_if_forward_ref(param_annotation, forward_refs_from_cls):
         and hasattr(forward_refs_from_cls, param_annotation)
     ):
         param_annotation = getattr(forward_refs_from_cls, param_annotation)
+        # Check if are in an annotation
+        param_annotation_origin = get_origin(param_annotation)
+        if param_annotation_origin in defintion_annotations_handlers:
+            annotations[
+                param_annotation_origin
+            ] = defintion_annotations_handlers[param_annotation_origin]
+            param_annotation = list(get_args(param_annotation))[0]
+        # Create definition
         if hasattr(param_annotation, "__name__") and hasattr(
             param_annotation, "__supertype__"
         ):
             # typing.NewType support
-            return new_type_to_defininition(param_annotation)
+            definition = new_type_to_defininition(param_annotation)
+            for handler in annotations.values():
+                handler(definition)
+            return definition
     return param_annotation
 
 
@@ -118,6 +152,7 @@ def _create_definition(
     default=NO_DEFAULT,
     *,
     forward_refs_from_cls: Optional[object] = None,
+    lock: bool = False,
 ):
     param_annotation = resolve_if_forward_ref(
         param_annotation, forward_refs_from_cls
@@ -138,12 +173,14 @@ def _create_definition(
     elif get_origin(param_annotation) in [
         Union,
         collections.abc.AsyncIterator,
+        LockReadWrite,
     ]:
         # If the annotation is of the form Optional
         return create_definition(
             name,
             list(get_args(param_annotation))[0],
             forward_refs_from_cls=forward_refs_from_cls,
+            lock=bool(get_origin(param_annotation) in (LockReadWrite,),),
         )
     elif (
         get_origin(param_annotation) is list
@@ -235,6 +272,7 @@ def create_definition(
     default=NO_DEFAULT,
     *,
     forward_refs_from_cls: Optional[object] = None,
+    lock: bool = False,
 ):
     if hasattr(param_annotation, "__name__") and hasattr(
         param_annotation, "__supertype__"
@@ -246,6 +284,7 @@ def create_definition(
         param_annotation,
         default=default,
         forward_refs_from_cls=forward_refs_from_cls,
+        lock=lock,
     )
     # We can guess name if converting from NewType. However, we can't otherwise.
     if not definition.name:
@@ -847,7 +886,9 @@ class DataFlow:
         for operation in args:
             name = getattr(getattr(operation, "op", operation), "name")
             if name in operations:
-                raise ValueError(f"Operation {name} given as positional and in dict")
+                raise ValueError(
+                    f"Operation {name} given as positional and in dict"
+                )
             operations[name] = operation
 
         self.operations = operations
diff --git a/entities/alice/alice/please/contribute/recommended_community_standards/recommended_community_standards.py b/entities/alice/alice/please/contribute/recommended_community_standards/recommended_community_standards.py
index 825f949d65..0ff7e11c31 100644
--- a/entities/alice/alice/please/contribute/recommended_community_standards/recommended_community_standards.py
+++ b/entities/alice/alice/please/contribute/recommended_community_standards/recommended_community_standards.py
@@ -8,18 +8,21 @@ import dffml
 import dffml_feature_git.feature.definitions
 
 
-class AliceGitRepo(NamedTuple):
+class AliceGitRepoSpec(NamedTuple):
     directory: str
     URL: str
 
 
+AliceGitRepo = dffml.LockReadWrite[AliceGitRepoSpec]
+
+
 class AliceGitRepoInputSetContextHandle(dffml.BaseContextHandle):
     def as_string(self) -> str:
         return str(self.ctx.repo)
 
 
 class AliceGitRepoInputSetContext(dffml.BaseInputSetContext):
-    def __init__(self, repo: AliceGitRepo):
+    def __init__(self, repo: AliceGitRepoSpec):
         self.repo = repo
 
     async def handle(self) -> AliceGitRepoInputSetContextHandle:

Is this the same as what we had in c89d3d8 ?

    async def alice_contribute_readme(self, repo: AliceGitRepo) -> ReadmeGitRepo:
        key, definition = list(self.parent.op.outputs.items())[0]
        await self.octx.ictx.cadd(
            AliceGitRepoInputSetContext(repo),
            dffml.Input(
                value=repo,
                definition=definition,
                parents=None,
                origin=(self.parent.op.instance_name, key),
            )
        )

diff --git a/entities/alice/alice/please/contribute/recommended_community_standards/recommended_community_standards.py b/entities/alice/alice/please/contribute/recommended_community_standards/recommended_community_standards.py
index 825f949d65..1bc1c41e50 100644
--- a/entities/alice/alice/please/contribute/recommended_community_standards/recommended_community_standards.py
+++ b/entities/alice/alice/please/contribute/recommended_community_standards/recommended_community_standards.py
@@ -203,30 +203,22 @@ class OverlayREADME:
     ReadmePRBody = NewType("github.pr.body", str)

     # async def cli_run_on_repo(self, repo: "CLIRunOnRepo"):
-    async def alice_contribute_readme(self, repo: AliceGitRepo) -> ReadmeGitRepo:
-        # TODO Clean this up once SystemContext refactor complete
-        readme_dataflow_cls_upstream = OverlayREADME
-        readme_dataflow_cls_overlays = dffml.Overlay.load(
-            entrypoint="dffml.overlays.alice.please.contribute.recommended_community_standards.overlay.readme"
-        )
-        readme_dataflow_upstream = dffml.DataFlow(
-            *dffml.object_to_operations(readme_dataflow_cls_upstream)
-        )
+    async def new_context(self, repo: AliceGitRepo) -> ReadmeGitRepo:
+        return
         # auto_flow with overlays
-        readme_dataflow = dffml.DataFlow(
+        dataflow = dffml.DataFlow(
             *itertools.chain(
                 *[
                     dffml.object_to_operations(cls)
                     for cls in [
-                        readme_dataflow_cls_upstream,
-                        *readme_dataflow_cls_overlays,
+                        upstream,
+                        *overlays,
                     ]
                 ]
             )
         )
         async with dffml.run_dataflow.imp(
-            # dataflow=self.octx.config.dataflow,
-            dataflow=readme_dataflow,
+            dataflow=dataflow,
             input_set_context_cls=AliceGitRepoInputSetContext,
         ) as custom_run_dataflow:
             # Copy all inputs from parent context into child. We eventually
@@ -277,6 +269,18 @@ class OverlayREADME:
                     },
                 )

+    async def alice_contribute_readme(self, repo: AliceGitRepo) -> ReadmeGitRepo:
+        key, definition = list(self.parent.op.outputs.items())[0]
+        await self.octx.ictx.cadd(
+            AliceGitRepoInputSetContext(repo),
+            dffml.Input(
+                value=repo,
+                definition=definition,
+                parents=None,
+                origin=(self.parent.op.instance_name, key),
+            )
+        )
+
     # TODO Run this system context where readme contexts is given on CLI or
     # overriden via disabling of static overlay and application of overlay to
     # generate contents dynamiclly.

Visualize the flow before we attempt to add CONTRIBUTING.md contribution

$ dffml service dev export alice.cli:AlicePleaseContributeCLIDataFlow | tee alice.please.contribute.recommended_community_standards.json
$ (echo -e 'HTTP/1.0 200 OK\n' && dffml dataflow diagram -shortname alice.please.contribute.recommended_community_standards.json) | nc -Nlp 9999;

graph TD
subgraph a759a07029077edc5c37fea0326fa281[Processing Stage]
style a759a07029077edc5c37fea0326fa281 fill:#afd388b5,stroke:#a4ca7a
subgraph 8cfb8cd5b8620de4a7ebe0dfec00771a[cli_has_repos]
style 8cfb8cd5b8620de4a7ebe0dfec00771a fill:#fff4de,stroke:#cece71
d493c90433d19f11f33c2d72cd144940[cli_has_repos]
e07552ee3b6b7696cb3ddd786222eaad(cmd)
e07552ee3b6b7696cb3ddd786222eaad --> d493c90433d19f11f33c2d72cd144940
cee6b5fdd0b6fbd0539cdcdc7f5a3324(wanted)
cee6b5fdd0b6fbd0539cdcdc7f5a3324 --> d493c90433d19f11f33c2d72cd144940
79e1ea6822bff603a835fb8ee80c7ff3(result)
d493c90433d19f11f33c2d72cd144940 --> 79e1ea6822bff603a835fb8ee80c7ff3
end
subgraph 0c2b64320fb5666a034794bb2195ecf0[cli_is_asking_for_recommended_community_standards]
style 0c2b64320fb5666a034794bb2195ecf0 fill:#fff4de,stroke:#cece71
222ee6c0209f1f1b7a782bc1276868c7[cli_is_asking_for_recommended_community_standards]
330f463830aa97e88917d5a9d1c21500(cmd)
330f463830aa97e88917d5a9d1c21500 --> 222ee6c0209f1f1b7a782bc1276868c7
ba29b52e9c5aa88ea1caeeff29bfd491(result)
222ee6c0209f1f1b7a782bc1276868c7 --> ba29b52e9c5aa88ea1caeeff29bfd491
end
subgraph eac58e8db2b55cb9cc5474aaa402c93e[cli_is_meant_on_this_repo]
style eac58e8db2b55cb9cc5474aaa402c93e fill:#fff4de,stroke:#cece71
6c819ad0228b0e7094b33e0634da9a38[cli_is_meant_on_this_repo]
dc7c5f0836f7d2564c402bf956722672(cmd)
dc7c5f0836f7d2564c402bf956722672 --> 6c819ad0228b0e7094b33e0634da9a38
58d8518cb0d6ef6ad35dc242486f1beb(wanted)
58d8518cb0d6ef6ad35dc242486f1beb --> 6c819ad0228b0e7094b33e0634da9a38
135ee61e3402d6fcbd7a219b0b4ccd73(result)
6c819ad0228b0e7094b33e0634da9a38 --> 135ee61e3402d6fcbd7a219b0b4ccd73
end
subgraph 37887bf260c5c8e9bd18038401008bbc[cli_run_on_repo]
style 37887bf260c5c8e9bd18038401008bbc fill:#fff4de,stroke:#cece71
9d1042f33352800e54d98c9c5a4223df[cli_run_on_repo]
e824ae072860bc545fc7d55aa0bca479(repo)
e824ae072860bc545fc7d55aa0bca479 --> 9d1042f33352800e54d98c9c5a4223df
40109d487bb9f08608d8c5f6e747042f(result)
9d1042f33352800e54d98c9c5a4223df --> 40109d487bb9f08608d8c5f6e747042f
end
subgraph 66ecd0c1f2e08941c443ec9cd89ec589[guess_repo_string_is_directory]
style 66ecd0c1f2e08941c443ec9cd89ec589 fill:#fff4de,stroke:#cece71
737d719a0c348ff65456024ddbc530fe[guess_repo_string_is_directory]
33d806f9b732bfd6b96ae2e9e4243a68(repo_string)
33d806f9b732bfd6b96ae2e9e4243a68 --> 737d719a0c348ff65456024ddbc530fe
dd5aab190ce844673819298c5b8fde76(result)
737d719a0c348ff65456024ddbc530fe --> dd5aab190ce844673819298c5b8fde76
end
subgraph 2bcd191634373f4b97ecb9546df23ee5[alice_contribute_contributing]
style 2bcd191634373f4b97ecb9546df23ee5 fill:#fff4de,stroke:#cece71
a2541ce40b2e5453e8e919021011e5e4[alice_contribute_contributing]
3786b4af914402320d260d077844620e(repo)
3786b4af914402320d260d077844620e --> a2541ce40b2e5453e8e919021011e5e4
da4270ecc44b6d9eed9809a560d24a28(result)
a2541ce40b2e5453e8e919021011e5e4 --> da4270ecc44b6d9eed9809a560d24a28
end
subgraph 13b430e6b93de7e40957165687f8e593[contribute_contributing_md]
style 13b430e6b93de7e40957165687f8e593 fill:#fff4de,stroke:#cece71
ff8f8968322872ccc3cf151d167e22a2[contribute_contributing_md]
4f752ce18209f62ed749e88dd1f70266(base)
4f752ce18209f62ed749e88dd1f70266 --> ff8f8968322872ccc3cf151d167e22a2
2def8c6923c832adf33989b26c91295a(commit_message)
2def8c6923c832adf33989b26c91295a --> ff8f8968322872ccc3cf151d167e22a2
f5548fcbcec8745ddf04104fc78e83a3(repo)
f5548fcbcec8745ddf04104fc78e83a3 --> ff8f8968322872ccc3cf151d167e22a2
24292ae12efd27a227a0d6368ba01faa(result)
ff8f8968322872ccc3cf151d167e22a2 --> 24292ae12efd27a227a0d6368ba01faa
end
subgraph 71a5f33f393735fa1cc91419b43db115[contributing_commit_message]
style 71a5f33f393735fa1cc91419b43db115 fill:#fff4de,stroke:#cece71
d034a42488583464e601bcaee619a539[contributing_commit_message]
c0a0fa68a872adf890ed639e07ed5882(issue_url)
c0a0fa68a872adf890ed639e07ed5882 --> d034a42488583464e601bcaee619a539
ce14ca2191f2b1c13c605b240e797255(result)
d034a42488583464e601bcaee619a539 --> ce14ca2191f2b1c13c605b240e797255
end
subgraph db8a1253cc59982323848f5e42c23c9d[contributing_issue]
style db8a1253cc59982323848f5e42c23c9d fill:#fff4de,stroke:#cece71
c39bd2cc88723432048c434fdd337eab[contributing_issue]
821d21e8a69d1fa1757147e7e768f306(body)
821d21e8a69d1fa1757147e7e768f306 --> c39bd2cc88723432048c434fdd337eab
0581b90c76b0a4635a968682b060abff(repo)
0581b90c76b0a4635a968682b060abff --> c39bd2cc88723432048c434fdd337eab
809719538467f6d0bf18f7ae26f08d80(title)
809719538467f6d0bf18f7ae26f08d80 --> c39bd2cc88723432048c434fdd337eab
c9f2ea5a7f25b3ae9fbf5041be5fa071(result)
c39bd2cc88723432048c434fdd337eab --> c9f2ea5a7f25b3ae9fbf5041be5fa071
end
subgraph 1e6046d1a567bf390566b1b995df9dcf[contributing_pr]
style 1e6046d1a567bf390566b1b995df9dcf fill:#fff4de,stroke:#cece71
4ec1433342f2f12ab8c59efab20e7b06[contributing_pr]
bb85c3467b05192c99a3954968c7a612(base)
bb85c3467b05192c99a3954968c7a612 --> 4ec1433342f2f12ab8c59efab20e7b06
77f6c1c6b7ee62881b49c289097dfbde(body)
77f6c1c6b7ee62881b49c289097dfbde --> 4ec1433342f2f12ab8c59efab20e7b06
a0a2fabc65fe5601c7ea289124d04f70(head)
a0a2fabc65fe5601c7ea289124d04f70 --> 4ec1433342f2f12ab8c59efab20e7b06
cf92708915b9f41cb490b991abd6c374(origin)
cf92708915b9f41cb490b991abd6c374 --> 4ec1433342f2f12ab8c59efab20e7b06
210ae36c85f3597c248e0b32da7661ae(repo)
210ae36c85f3597c248e0b32da7661ae --> 4ec1433342f2f12ab8c59efab20e7b06
1700dc637c25bd503077a2a1422142e2(title)
1700dc637c25bd503077a2a1422142e2 --> 4ec1433342f2f12ab8c59efab20e7b06
806e8c455d2bb7ad68112d2a7e16eed6(result)
4ec1433342f2f12ab8c59efab20e7b06 --> 806e8c455d2bb7ad68112d2a7e16eed6
end
subgraph 04c27c13241164ae88456c1377995897[contributing_pr_body]
style 04c27c13241164ae88456c1377995897 fill:#fff4de,stroke:#cece71
a3cebe78451142664930d44ad4d7d181[contributing_pr_body]
6118470d0158ef1a220fe7c7232e1b63(contributing_issue)
6118470d0158ef1a220fe7c7232e1b63 --> a3cebe78451142664930d44ad4d7d181
99a7dd1ae037153eef80e1dee51b9d2b(result)
a3cebe78451142664930d44ad4d7d181 --> 99a7dd1ae037153eef80e1dee51b9d2b
end
subgraph 0d4627f8d8564b6c4ba33c12dcb58fc1[contributing_pr_title]
style 0d4627f8d8564b6c4ba33c12dcb58fc1 fill:#fff4de,stroke:#cece71
bfa172a9399604546048d60db0a36187[contributing_pr_title]
0fd26f9166ccca10c68e9aefa9c15767(contributing_issue)
0fd26f9166ccca10c68e9aefa9c15767 --> bfa172a9399604546048d60db0a36187
77a2f9d4dfad5f520f1502e8ba70e47a(result)
bfa172a9399604546048d60db0a36187 --> 77a2f9d4dfad5f520f1502e8ba70e47a
end
subgraph c67b92ef6a2e025ca086bc2f89d9afbb[create_contributing_file_if_not_exists]
style c67b92ef6a2e025ca086bc2f89d9afbb fill:#fff4de,stroke:#cece71
993a1fe069a02a45ba3579b1902b2a36[create_contributing_file_if_not_exists]
401c179bb30b24c2ca989c64d0b1cdc7(contributing_contents)
401c179bb30b24c2ca989c64d0b1cdc7 --> 993a1fe069a02a45ba3579b1902b2a36
dde78f81b1bdfe02c0a2bf6e51f65cb4(repo)
dde78f81b1bdfe02c0a2bf6e51f65cb4 --> 993a1fe069a02a45ba3579b1902b2a36
e5b8d158dc0ec476dbbd44549a981815(result)
993a1fe069a02a45ba3579b1902b2a36 --> e5b8d158dc0ec476dbbd44549a981815
end
subgraph 4ea6696419c4a0862a4f63ea1f60c751[create_branch_if_none_exists]
style 4ea6696419c4a0862a4f63ea1f60c751 fill:#fff4de,stroke:#cece71
502369b37882b300d6620d5b4020f5b2[create_branch_if_none_exists]
fdcb9b6113856222e30e093f7c38065e(name)
fdcb9b6113856222e30e093f7c38065e --> 502369b37882b300d6620d5b4020f5b2
bdcf4b078985f4a390e4ed4beacffa65(repo)
bdcf4b078985f4a390e4ed4beacffa65 --> 502369b37882b300d6620d5b4020f5b2
5a5493ab86ab4053f1d44302e7bdddd6(result)
502369b37882b300d6620d5b4020f5b2 --> 5a5493ab86ab4053f1d44302e7bdddd6
end
subgraph b1d510183f6a4c3fde207a4656c72cb4[determin_base_branch]
style b1d510183f6a4c3fde207a4656c72cb4 fill:#fff4de,stroke:#cece71
476aecd4d4d712cda1879feba46ea109[determin_base_branch]
ff47cf65b58262acec28507f4427de45(default_branch)
ff47cf65b58262acec28507f4427de45 --> 476aecd4d4d712cda1879feba46ea109
150204cd2d5a921deb53c312418379a1(result)
476aecd4d4d712cda1879feba46ea109 --> 150204cd2d5a921deb53c312418379a1
end
subgraph 2a08ff341f159c170b7fe017eaad2f18[git_repo_to_alice_git_repo]
style 2a08ff341f159c170b7fe017eaad2f18 fill:#fff4de,stroke:#cece71
7f74112f6d30c6289caa0a000e87edab[git_repo_to_alice_git_repo]
e58180baf478fe910359358a3fa02234(repo)
e58180baf478fe910359358a3fa02234 --> 7f74112f6d30c6289caa0a000e87edab
9b92d5a346885079a2821c4d27cb5174(result)
7f74112f6d30c6289caa0a000e87edab --> 9b92d5a346885079a2821c4d27cb5174
end
subgraph b5d35aa8a8dcd28d22d47caad02676b0[guess_repo_string_is_url]
style b5d35aa8a8dcd28d22d47caad02676b0 fill:#fff4de,stroke:#cece71
0de074e71a32e30889b8bb400cf8db9f[guess_repo_string_is_url]
c3bfe79b396a98ce2d9bfe772c9c20af(repo_string)
c3bfe79b396a98ce2d9bfe772c9c20af --> 0de074e71a32e30889b8bb400cf8db9f
2a1c620b0d510c3d8ed35deda41851c5(result)
0de074e71a32e30889b8bb400cf8db9f --> 2a1c620b0d510c3d8ed35deda41851c5
end
subgraph 60791520c6d124c0bf15e599132b0caf[guessed_repo_string_is_operations_git_url]
style 60791520c6d124c0bf15e599132b0caf fill:#fff4de,stroke:#cece71
102f173505d7b546236cdeff191369d4[guessed_repo_string_is_operations_git_url]
4934c6211334318c63a5e91530171c9b(repo_url)
4934c6211334318c63a5e91530171c9b --> 102f173505d7b546236cdeff191369d4
8d0adc31da1a0919724baf73d047743c(result)
102f173505d7b546236cdeff191369d4 --> 8d0adc31da1a0919724baf73d047743c
end
subgraph f2c7b93622447999daab403713239ada[guessed_repo_string_means_no_git_branch_given]
style f2c7b93622447999daab403713239ada fill:#fff4de,stroke:#cece71
c8294a87e7aae8f7f9cb7f53e054fed5[guessed_repo_string_means_no_git_branch_given]
5567dd8a6d7ae4fe86252db32e189a4d(repo_url)
5567dd8a6d7ae4fe86252db32e189a4d --> c8294a87e7aae8f7f9cb7f53e054fed5
d888e6b64b5e3496056088f14dab9894(result)
c8294a87e7aae8f7f9cb7f53e054fed5 --> d888e6b64b5e3496056088f14dab9894
end
subgraph 113addf4beee5305fdc79d2363608f9d[github_owns_remote]
style 113addf4beee5305fdc79d2363608f9d fill:#fff4de,stroke:#cece71
049b72b81b976fbb43607bfeeb0464c5[github_owns_remote]
6c2b36393ffff6be0b4ad333df2d9419(remote)
6c2b36393ffff6be0b4ad333df2d9419 --> 049b72b81b976fbb43607bfeeb0464c5
19a9ee483c1743e6ecf0a2dc3b6f8c7a(repo)
19a9ee483c1743e6ecf0a2dc3b6f8c7a --> 049b72b81b976fbb43607bfeeb0464c5
b4cff8d194413f436d94f9d84ece0262(result)
049b72b81b976fbb43607bfeeb0464c5 --> b4cff8d194413f436d94f9d84ece0262
end
subgraph 8506cba6514466fb6d65f33ace4b0eac[alice_contribute_readme]
style 8506cba6514466fb6d65f33ace4b0eac fill:#fff4de,stroke:#cece71
d4507d3d1c3fbf3e7e373eae24797667[alice_contribute_readme]
68cf7d6869d027ca46a5fb4dbf7001d1(repo)
68cf7d6869d027ca46a5fb4dbf7001d1 --> d4507d3d1c3fbf3e7e373eae24797667
2f9316539862f119f7c525bf9061e974(result)
d4507d3d1c3fbf3e7e373eae24797667 --> 2f9316539862f119f7c525bf9061e974
end
subgraph 4233e6dc67cba131d4ef005af9c02959[contribute_readme_md]
style 4233e6dc67cba131d4ef005af9c02959 fill:#fff4de,stroke:#cece71
3db0ee5d6ab83886bded5afd86f3f88f[contribute_readme_md]
37044e4d8610abe13849bc71a5cb7591(base)
37044e4d8610abe13849bc71a5cb7591 --> 3db0ee5d6ab83886bded5afd86f3f88f
631c051fe6050ae8f8fc3321ed00802d(commit_message)
631c051fe6050ae8f8fc3321ed00802d --> 3db0ee5d6ab83886bded5afd86f3f88f
182194bab776fc9bc406ed573d621b68(repo)
182194bab776fc9bc406ed573d621b68 --> 3db0ee5d6ab83886bded5afd86f3f88f
0ee9f524d2db12be854fe611fa8126dd(result)
3db0ee5d6ab83886bded5afd86f3f88f --> 0ee9f524d2db12be854fe611fa8126dd
end
subgraph a6080d9c45eb5f806a47152a18bf7830[create_readme_file_if_not_exists]
style a6080d9c45eb5f806a47152a18bf7830 fill:#fff4de,stroke:#cece71
67e388f508dd96084c37d236a2c67e67[create_readme_file_if_not_exists]
54faf20bfdca0e63d07efb3e5a984cf1(readme_contents)
54faf20bfdca0e63d07efb3e5a984cf1 --> 67e388f508dd96084c37d236a2c67e67
8c089c362960ccf181742334a3dccaea(repo)
8c089c362960ccf181742334a3dccaea --> 67e388f508dd96084c37d236a2c67e67
5cc65e17d40e6a7223c1504f1c4b0d2a(result)
67e388f508dd96084c37d236a2c67e67 --> 5cc65e17d40e6a7223c1504f1c4b0d2a
end
subgraph e7757158127e9845b2915c16a7fa80c5[readme_commit_message]
style e7757158127e9845b2915c16a7fa80c5 fill:#fff4de,stroke:#cece71
562bdc535c7cebfc66dba920b1a17540[readme_commit_message]
0af5cbea9050874a0a3cba73bb61f892(issue_url)
0af5cbea9050874a0a3cba73bb61f892 --> 562bdc535c7cebfc66dba920b1a17540
2641f3b67327fb7518ee34a3a40b0755(result)
562bdc535c7cebfc66dba920b1a17540 --> 2641f3b67327fb7518ee34a3a40b0755
end
subgraph cf99ff6fad80e9c21266b43fd67b2f7b[readme_issue]
style cf99ff6fad80e9c21266b43fd67b2f7b fill:#fff4de,stroke:#cece71
da44417f891a945085590baafffc2bdb[readme_issue]
d519830ab4e07ec391038e8581889ac3(body)
d519830ab4e07ec391038e8581889ac3 --> da44417f891a945085590baafffc2bdb
268852aa3fa8ab0864a32abae5a333f7(repo)
268852aa3fa8ab0864a32abae5a333f7 --> da44417f891a945085590baafffc2bdb
77a11dd29af309cf43ed321446c4bf01(title)
77a11dd29af309cf43ed321446c4bf01 --> da44417f891a945085590baafffc2bdb
1d2360c9da18fac0b6ec142df8f3fbda(result)
da44417f891a945085590baafffc2bdb --> 1d2360c9da18fac0b6ec142df8f3fbda
end
subgraph 7ec0442cf2d95c367912e8abee09b217[readme_pr]
style 7ec0442cf2d95c367912e8abee09b217 fill:#fff4de,stroke:#cece71
bb314dc452cde5b6af5ea94dd277ba40[readme_pr]
127d77c3047facc1daa621148c5a0a1d(base)
127d77c3047facc1daa621148c5a0a1d --> bb314dc452cde5b6af5ea94dd277ba40
cb421e4de153cbb912f7fbe57e4ad734(body)
cb421e4de153cbb912f7fbe57e4ad734 --> bb314dc452cde5b6af5ea94dd277ba40
cbf7a0b88c0a41953b245303f3e9a0d3(head)
cbf7a0b88c0a41953b245303f3e9a0d3 --> bb314dc452cde5b6af5ea94dd277ba40
e5f9ad44448abd2469b3fd9831f3d159(origin)
e5f9ad44448abd2469b3fd9831f3d159 --> bb314dc452cde5b6af5ea94dd277ba40
a35aee6711d240378eb57a3932537ca1(repo)
a35aee6711d240378eb57a3932537ca1 --> bb314dc452cde5b6af5ea94dd277ba40
dfcce88a7d605d46bf17de1159fbe5ad(title)
dfcce88a7d605d46bf17de1159fbe5ad --> bb314dc452cde5b6af5ea94dd277ba40
a210a7890a7bea8d629368e02da3d806(result)
bb314dc452cde5b6af5ea94dd277ba40 --> a210a7890a7bea8d629368e02da3d806
end
subgraph 227eabb1f1c5cc0bc931714a03049e27[readme_pr_body]
style 227eabb1f1c5cc0bc931714a03049e27 fill:#fff4de,stroke:#cece71
2aea976396cfe68dacd9bc7d4a3f0cba[readme_pr_body]
c5dfd309617c909b852afe0b4ae4a178(readme_issue)
c5dfd309617c909b852afe0b4ae4a178 --> 2aea976396cfe68dacd9bc7d4a3f0cba
40ddb5b508cb5643e7c91f7abdb72b84(result)
2aea976396cfe68dacd9bc7d4a3f0cba --> 40ddb5b508cb5643e7c91f7abdb72b84
end
subgraph 48687c84e69b3db0acca625cbe2e6b49[readme_pr_title]
style 48687c84e69b3db0acca625cbe2e6b49 fill:#fff4de,stroke:#cece71
d8668ff93f41bc241c8c540199cd7453[readme_pr_title]
3b2137dd1c61d0dac7d4e40fd6746cfb(readme_issue)
3b2137dd1c61d0dac7d4e40fd6746cfb --> d8668ff93f41bc241c8c540199cd7453
956e024fde513b3a449eac9ee42d6ab3(result)
d8668ff93f41bc241c8c540199cd7453 --> 956e024fde513b3a449eac9ee42d6ab3
end
subgraph d3ec0ac85209a7256c89d20f758f09f4[check_if_valid_git_repository_URL]
style d3ec0ac85209a7256c89d20f758f09f4 fill:#fff4de,stroke:#cece71
f577c71443f6b04596b3fe0511326c40[check_if_valid_git_repository_URL]
7440e73a8e8f864097f42162b74f2762(URL)
7440e73a8e8f864097f42162b74f2762 --> f577c71443f6b04596b3fe0511326c40
8e39b501b41c5d0e4596318f80a03210(valid)
f577c71443f6b04596b3fe0511326c40 --> 8e39b501b41c5d0e4596318f80a03210
end
subgraph af8da22d1318d911f29b95e687f87c5d[clone_git_repo]
style af8da22d1318d911f29b95e687f87c5d fill:#fff4de,stroke:#cece71
155b8fdb5524f6bfd5adbae4940ad8d5[clone_git_repo]
eed77b9eea541e0c378c67395351099c(URL)
eed77b9eea541e0c378c67395351099c --> 155b8fdb5524f6bfd5adbae4940ad8d5
8b5928cd265dd2c44d67d076f60c8b05(ssh_key)
8b5928cd265dd2c44d67d076f60c8b05 --> 155b8fdb5524f6bfd5adbae4940ad8d5
4e1d5ea96e050e46ebf95ebc0713d54c(repo)
155b8fdb5524f6bfd5adbae4940ad8d5 --> 4e1d5ea96e050e46ebf95ebc0713d54c
6a44de06a4a3518b939b27c790f6cdce{valid_git_repository_URL}
6a44de06a4a3518b939b27c790f6cdce --> 155b8fdb5524f6bfd5adbae4940ad8d5
end
subgraph d3d91578caf34c0ae944b17853783406[git_repo_default_branch]
style d3d91578caf34c0ae944b17853783406 fill:#fff4de,stroke:#cece71
546062a96122df465d2631f31df4e9e3[git_repo_default_branch]
181f1b33df4d795fbad2911ec7087e86(repo)
181f1b33df4d795fbad2911ec7087e86 --> 546062a96122df465d2631f31df4e9e3
57651c1bcd24b794dfc8d1794ab556d5(branch)
546062a96122df465d2631f31df4e9e3 --> 57651c1bcd24b794dfc8d1794ab556d5
5ed1ab77e726d7efdcc41e9e2f8039c6(remote)
546062a96122df465d2631f31df4e9e3 --> 5ed1ab77e726d7efdcc41e9e2f8039c6
4c3cdd5f15b7a846d291aac089e8a622{no_git_branch_given}
4c3cdd5f15b7a846d291aac089e8a622 --> 546062a96122df465d2631f31df4e9e3
end
end
subgraph a4827add25f5c7d5895c5728b74e2beb[Cleanup Stage]
style a4827add25f5c7d5895c5728b74e2beb fill:#afd388b5,stroke:#a4ca7a
end
subgraph 58ca4d24d2767176f196436c2890b926[Output Stage]
style 58ca4d24d2767176f196436c2890b926 fill:#afd388b5,stroke:#a4ca7a
end
subgraph inputs[Inputs]
style inputs fill:#f6dbf9,stroke:#a178ca
128516cfa09b0383023eab52ee24878a(seed<br>dffml.util.cli.CMD)
128516cfa09b0383023eab52ee24878a --> e07552ee3b6b7696cb3ddd786222eaad
ba29b52e9c5aa88ea1caeeff29bfd491 --> cee6b5fdd0b6fbd0539cdcdc7f5a3324
128516cfa09b0383023eab52ee24878a(seed<br>dffml.util.cli.CMD)
128516cfa09b0383023eab52ee24878a --> 330f463830aa97e88917d5a9d1c21500
128516cfa09b0383023eab52ee24878a(seed<br>dffml.util.cli.CMD)
128516cfa09b0383023eab52ee24878a --> dc7c5f0836f7d2564c402bf956722672
ba29b52e9c5aa88ea1caeeff29bfd491 --> 58d8518cb0d6ef6ad35dc242486f1beb
79e1ea6822bff603a835fb8ee80c7ff3 --> e824ae072860bc545fc7d55aa0bca479
135ee61e3402d6fcbd7a219b0b4ccd73 --> e824ae072860bc545fc7d55aa0bca479
40109d487bb9f08608d8c5f6e747042f --> 33d806f9b732bfd6b96ae2e9e4243a68
dd5aab190ce844673819298c5b8fde76 --> 3786b4af914402320d260d077844620e
9b92d5a346885079a2821c4d27cb5174 --> 3786b4af914402320d260d077844620e
150204cd2d5a921deb53c312418379a1 --> 4f752ce18209f62ed749e88dd1f70266
ce14ca2191f2b1c13c605b240e797255 --> 2def8c6923c832adf33989b26c91295a
da4270ecc44b6d9eed9809a560d24a28 --> f5548fcbcec8745ddf04104fc78e83a3
c9f2ea5a7f25b3ae9fbf5041be5fa071 --> c0a0fa68a872adf890ed639e07ed5882
c94383981c3a071b8c3df7293c8c7c92(seed<br>ContributingIssueBody)
c94383981c3a071b8c3df7293c8c7c92 --> 821d21e8a69d1fa1757147e7e768f306
da4270ecc44b6d9eed9809a560d24a28 --> 0581b90c76b0a4635a968682b060abff
90c6a88275f27b28dc12f5741ac1652f(seed<br>ContributingIssueTitle)
90c6a88275f27b28dc12f5741ac1652f --> 809719538467f6d0bf18f7ae26f08d80
150204cd2d5a921deb53c312418379a1 --> bb85c3467b05192c99a3954968c7a612
99a7dd1ae037153eef80e1dee51b9d2b --> 77f6c1c6b7ee62881b49c289097dfbde
24292ae12efd27a227a0d6368ba01faa --> a0a2fabc65fe5601c7ea289124d04f70
b4cff8d194413f436d94f9d84ece0262 --> cf92708915b9f41cb490b991abd6c374
da4270ecc44b6d9eed9809a560d24a28 --> 210ae36c85f3597c248e0b32da7661ae
77a2f9d4dfad5f520f1502e8ba70e47a --> 1700dc637c25bd503077a2a1422142e2
c9f2ea5a7f25b3ae9fbf5041be5fa071 --> 6118470d0158ef1a220fe7c7232e1b63
c9f2ea5a7f25b3ae9fbf5041be5fa071 --> 0fd26f9166ccca10c68e9aefa9c15767
90b3c16d6d8884aa6f70b475d98f661b(seed<br>repo.directory.contributing.contents)
90b3c16d6d8884aa6f70b475d98f661b --> 401c179bb30b24c2ca989c64d0b1cdc7
da4270ecc44b6d9eed9809a560d24a28 --> dde78f81b1bdfe02c0a2bf6e51f65cb4
21ccfd2c550bd853d28581f0b0c9f9fe(seed<br>default.branch.name)
21ccfd2c550bd853d28581f0b0c9f9fe --> fdcb9b6113856222e30e093f7c38065e
dd5aab190ce844673819298c5b8fde76 --> bdcf4b078985f4a390e4ed4beacffa65
9b92d5a346885079a2821c4d27cb5174 --> bdcf4b078985f4a390e4ed4beacffa65
5a5493ab86ab4053f1d44302e7bdddd6 --> ff47cf65b58262acec28507f4427de45
57651c1bcd24b794dfc8d1794ab556d5 --> ff47cf65b58262acec28507f4427de45
4e1d5ea96e050e46ebf95ebc0713d54c --> e58180baf478fe910359358a3fa02234
40109d487bb9f08608d8c5f6e747042f --> c3bfe79b396a98ce2d9bfe772c9c20af
2a1c620b0d510c3d8ed35deda41851c5 --> 4934c6211334318c63a5e91530171c9b
2a1c620b0d510c3d8ed35deda41851c5 --> 5567dd8a6d7ae4fe86252db32e189a4d
5ed1ab77e726d7efdcc41e9e2f8039c6 --> 6c2b36393ffff6be0b4ad333df2d9419
dd5aab190ce844673819298c5b8fde76 --> 19a9ee483c1743e6ecf0a2dc3b6f8c7a
9b92d5a346885079a2821c4d27cb5174 --> 19a9ee483c1743e6ecf0a2dc3b6f8c7a
dd5aab190ce844673819298c5b8fde76 --> 68cf7d6869d027ca46a5fb4dbf7001d1
9b92d5a346885079a2821c4d27cb5174 --> 68cf7d6869d027ca46a5fb4dbf7001d1
150204cd2d5a921deb53c312418379a1 --> 37044e4d8610abe13849bc71a5cb7591
2641f3b67327fb7518ee34a3a40b0755 --> 631c051fe6050ae8f8fc3321ed00802d
2f9316539862f119f7c525bf9061e974 --> 182194bab776fc9bc406ed573d621b68
d2708225c1f4c95d613a2645a17a5bc0(seed<br>repo.directory.readme.contents)
d2708225c1f4c95d613a2645a17a5bc0 --> 54faf20bfdca0e63d07efb3e5a984cf1
2f9316539862f119f7c525bf9061e974 --> 8c089c362960ccf181742334a3dccaea
1d2360c9da18fac0b6ec142df8f3fbda --> 0af5cbea9050874a0a3cba73bb61f892
1daacccd02f8117e67ad3cb8686a732c(seed<br>ReadmeIssueBody)
1daacccd02f8117e67ad3cb8686a732c --> d519830ab4e07ec391038e8581889ac3
2f9316539862f119f7c525bf9061e974 --> 268852aa3fa8ab0864a32abae5a333f7
0c1ab2d4bda10e1083557833ae5c5da4(seed<br>ReadmeIssueTitle)
0c1ab2d4bda10e1083557833ae5c5da4 --> 77a11dd29af309cf43ed321446c4bf01
150204cd2d5a921deb53c312418379a1 --> 127d77c3047facc1daa621148c5a0a1d
40ddb5b508cb5643e7c91f7abdb72b84 --> cb421e4de153cbb912f7fbe57e4ad734
0ee9f524d2db12be854fe611fa8126dd --> cbf7a0b88c0a41953b245303f3e9a0d3
b4cff8d194413f436d94f9d84ece0262 --> e5f9ad44448abd2469b3fd9831f3d159
2f9316539862f119f7c525bf9061e974 --> a35aee6711d240378eb57a3932537ca1
956e024fde513b3a449eac9ee42d6ab3 --> dfcce88a7d605d46bf17de1159fbe5ad
1d2360c9da18fac0b6ec142df8f3fbda --> c5dfd309617c909b852afe0b4ae4a178
1d2360c9da18fac0b6ec142df8f3fbda --> 3b2137dd1c61d0dac7d4e40fd6746cfb
8d0adc31da1a0919724baf73d047743c --> 7440e73a8e8f864097f42162b74f2762
8d0adc31da1a0919724baf73d047743c --> eed77b9eea541e0c378c67395351099c
a6ed501edbf561fda49a0a0a3ca310f0(seed<br>git_repo_ssh_key)
a6ed501edbf561fda49a0a0a3ca310f0 --> 8b5928cd265dd2c44d67d076f60c8b05
8e39b501b41c5d0e4596318f80a03210 --> 6a44de06a4a3518b939b27c790f6cdce
4e1d5ea96e050e46ebf95ebc0713d54c --> 181f1b33df4d795fbad2911ec7087e86
end

Notes
- create_*_if_not_exists doesn't appear connected.
Only either README or CONTRIBUTING is currently being added when
we run with our new CONTRIBUTING contribution flow overlayed.

$ for pr in $(gh -R https://github.com/pdxjohnny/testaaaa pr list --json number --jq '.[].number'); do gh -R https://github.com/pdxjohnny/testaaaa pr close "${pr}"; done
✓ Closed pull request #222 (Recommended Community Standard: README)
✓ Closed pull request #219 (Recommended Community Standard: CONTRIBUTING)
$ nodemon -e py --exec 'clear; for pr in $(gh -R https://github.com/pdxjohnny/testaaaa pr list --json number --jq '.[].number'); do gh -R https://github.com/pdxjohnny/testaaaa pr close "${pr}"; done; (alice please contribute -log debug -repos https://github.com/pdxjohnny/testaaaa -- recommended community standards; gh -R https://github.com/pdxjohnny/testaaaa pr list) 2>&1 | tee .output/$(date +%4Y-%m-%d-%H-%M).txt; test 1'
$ less -S .output/$(ls .output/ | tail -n 1)

Refactor into README and CONTRIBUTING Overlays

Had the thought, aren't we just adding a new context here?

diff --git a/dffml/df/memory.py b/dffml/df/memory.py
index 59286d4927..87c75d637b 100644
--- a/dffml/df/memory.py
+++ b/dffml/df/memory.py
@@ -377,6 +377,7 @@ class MemoryInputNetworkContext(BaseInputNetworkContext):
                     self.ctxhd[handle_string].by_origin[item.origin] = []
                 # Add input to by origin set
                 self.ctxhd[handle_string].by_origin[item.origin].append(item)
+                self.logger.debug("Added to %s: %r", handle_string, item)

     async def uadd(self, *args: Input):
         """
diff --git a/entities/alice/alice/please/contribute/recommended_community_standards/recommended_community_standards.py b/entities/alice/alice/please/contribute/recommended_community_standards/recommended_community_standards.py
index 2873a1b193..cc4d374e57 100644
--- a/entities/alice/alice/please/contribute/recommended_community_standards/recommended_community_standards.py
+++ b/entities/alice/alice/please/contribute/recommended_community_standards/recommended_community_standards.py
@@ -1,7 +1,8 @@
+import asyncio
 import pathlib
 import textwrap
 import itertools
-from typing import NamedTuple, NewType, Optional
+from typing import NamedTuple, NewType, Optional, Type, Any


 import dffml
@@ -183,6 +184,34 @@ class OverlayGitHub:
         return remote


+async def context_adder(
+    self,
+    upstream_cls: Type[Any],
+    input_set_context: dffml.BaseInputSetContext,
+    value: Any,
+):
+    upstream = dffml.DataFlow(*dffml.object_to_operations(upstream_cls))
+    key, definition = list(self.parent.op.outputs.items())[0]
+    async with self.octx.ictx.definitions(self.ctx) as definitions:
+        await self.octx.ictx.cadd(
+            input_set_context,
+            dffml.Input(
+                value=value,
+                definition=definition,
+                parents=None,
+                origin=(self.parent.op.instance_name, key),
+            ),
+            *[
+                item
+                async for item in definitions.inputs()
+                if (
+                    item.definition in upstream.definitions.values()
+                    and item.definition not in self.parent.op.inputs.values()
+                )
+            ],
+        )
+
+
 # NOTE Not sure if the orchestrator will know what to do if we do this
 # ReadmeGitRepo = AliceGitRepo
 class ReadmeGitRepo(NamedTuple):
@@ -204,6 +233,9 @@ class OverlayREADME:

     # async def cli_run_on_repo(self, repo: "CLIRunOnRepo"):
     async def alice_contribute_readme(self, repo: AliceGitRepo) -> ReadmeGitRepo:
+        # await context_adder(
+        #     self, OverlayREADME, AliceGitRepoInputSetContext(repo), repo
+        # )
         # TODO Clean this up once SystemContext refactor complete
         readme_dataflow_cls_upstream = OverlayREADME
         readme_dataflow_cls_overlays = dffml.Overlay.load(

DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGit:determin_base_branch Stage: PROCESSING: alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGit:determin_base_branch
DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGit:determin_base_branch Inputs: {'default_branch': 'master'}
DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGit:determin_base_branch Conditions: {}
DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGit:determin_base_branch Outputs: {'result': 'master'}
DEBUG:dffml.MemoryOperationImplementationNetworkContext:---
DEBUG:dffml.MemoryInputNetworkContext:Added to https://github.com/pdxjohnny/testaaaa: Input(value=master, definition=repo.git.base.branch)
DEBUG:dffml.MemoryLockNetworkContext:Acquiring: 6fc55525-c499-421c-8b07-497dd277b1ff(GitRepoSpec(directory='/tmp/dffml-feature-git-rrflb9gm', URL='https://github.com/pdxjohnny/testaaaa')) (now held by Operation(name='alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayCONTRIBUTING:alice_contribute_contributing', inputs={'repo': AliceGitRepo}, outputs={'result': ContributingGitRepo}, stage=<Stage.PROCESSING: 'processing'>, conditions=[], expand=[], instance_name='alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayCONTRIBUTING:alice_contribute_contributing', validator=False, retry=0))
DEBUG:dffml.MemoryOperationImplementationNetworkContext:---
DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayCONTRIBUTING:alice_contribute_contributing Stage: PROCESSING: alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayCONTRIBUTING:alice_contribute_contributing
DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayCONTRIBUTING:alice_contribute_contributing Inputs: {'repo': GitRepoSpec(directory='/tmp/dffml-feature-git-rrflb9gm', URL='https://github.com/pdxjohnny/testaaaa')}
DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayCONTRIBUTING:alice_contribute_contributing Conditions: {}
DEBUG:dffml.MemoryInputNetworkContext:Added to GitRepoSpec(directory='/tmp/dffml-feature-git-rrflb9gm', URL='https://github.com/pdxjohnny/testaaaa'): Input(value=GitRepoSpec(directory='/tmp/dffml-feature-git-rrflb9gm', URL='https://github.com/pdxjohnny/testaaaa'), definition=ContributingGitRepo)
DEBUG:dffml.MemoryInputNetworkContext:Added to GitRepoSpec(directory='/tmp/dffml-feature-git-rrflb9gm', URL='https://github.com/pdxjohnny/testaaaa'): Input(value=origin, definition=writable.github.remote.origin)
DEBUG:dffml.MemoryInputNetworkContext:Added to GitRepoSpec(directory='/tmp/dffml-feature-git-rrflb9gm', URL='https://github.com/pdxjohnny/testaaaa'): Input(value=master, definition=repo.git.base.branch)
DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayCONTRIBUTING:alice_contribute_contributing Outputs: None
DEBUG:dffml.MemoryOperationImplementationNetworkContext:---
DEBUG:dffml.MemoryLockNetworkContext:Acquiring: 6fc55525-c499-421c-8b07-497dd277b1ff(GitRepoSpec(directory='/tmp/dffml-feature-git-rrflb9gm', URL='https://github.com/pdxjohnny/testaaaa')) (now held by Operation(name='alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGit:create_branch_if_none_exists', inputs={'repo': AliceGitRepo, 'name': default.branch.name}, outputs={'result': git_branch}, stage=<Stage.PROCESSING: 'processing'>, conditions=[], expand=[], instance_name='alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGit:create_branch_if_none_exists', validator=False, retry=0))
DEBUG:dffml.MemoryOperationImplementationNetworkContext:---
DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGit:create_branch_if_none_exists Stage: PROCESSING: alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGit:create_branch_if_none_exists
DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGit:create_branch_if_none_exists Inputs: {'repo': GitRepoSpec(directory='/tmp/dffml-feature-git-rrflb9gm', URL='https://github.com/pdxjohnny/testaaaa'), 'name': 'main'}
DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGit:create_branch_if_none_exists Conditions: {}
DEBUG:dffml_feature_git.util:proc.create: ('git', 'branch', '-r')
DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGit:create_branch_if_none_exists Outputs: None
DEBUG:dffml.MemoryOperationImplementationNetworkContext:---
DEBUG:dffml.MemoryLockNetworkContext:Acquiring: 6fc55525-c499-421c-8b07-497dd277b1ff(GitRepoSpec(directory='/tmp/dffml-feature-git-rrflb9gm', URL='https://github.com/pdxjohnny/testaaaa')) (now held by Operation(name='alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:alice_contribute_readme', inputs={'repo': AliceGitRepo}, outputs={'result': ReadmeGitRepo}, stage=<Stage.PROCESSING: 'processing'>, conditions=[], expand=[], instance_name='alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:alice_contribute_readme', validator=False, retry=0))
DEBUG:dffml.MemoryOperationImplementationNetworkContext:---
DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:alice_contribute_readme Stage: PROCESSING: alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:alice_contribute_readme
DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:alice_contribute_readme Inputs: {'repo': GitRepoSpec(directory='/tmp/dffml-feature-git-rrflb9gm', URL='https://github.com/pdxjohnny/testaaaa')}
DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:alice_contribute_readme Conditions: {}
DEBUG:dffml.MemoryInputNetworkContext:Added to GitRepoSpec(directory='/tmp/dffml-feature-git-rrflb9gm', URL='https://github.com/pdxjohnny/testaaaa'): Input(value=GitRepoSpec(directory='/tmp/dffml-feature-git-rrflb9gm', URL='https://github.com/pdxjohnny/testaaaa'), definition=ReadmeGitRepo)
DEBUG:dffml.MemoryInputNetworkContext:Added to GitRepoSpec(directory='/tmp/dffml-feature-git-rrflb9gm', URL='https://github.com/pdxjohnny/testaaaa'): Input(value=origin, definition=writable.github.remote.origin)
DEBUG:dffml.MemoryInputNetworkContext:Added to GitRepoSpec(directory='/tmp/dffml-feature-git-rrflb9gm', URL='https://github.com/pdxjohnny/testaaaa'): Input(value=master, definition=repo.git.base.branch)
DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:alice_contribute_readme Outputs: None
DEBUG:dffml.MemoryOperationImplementationNetworkContext:---
DEBUG:dffml.MemoryInputNetworkContext:Received https://github.com/pdxjohnny/testaaaa result {} from <dffml.df.memory.MemoryOrchestratorContext object at 0x7f2405795d90>
DEBUG:dffml.MemoryInputNetworkContext:Received https://github.com/pdxjohnny/testaaaa result {} from <dffml.df.memory.MemoryOrchestratorContext object at 0x7f2405795d90>
DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.cli.OverlayCLI:cli_run_on_repo Outputs: None
DEBUG:dffml.MemoryOperationImplementationNetworkContext:---
DEBUG:dffml.MemoryOrchestratorContext:ctx.outstanding: 1
DEBUG:dffml.MemoryInputNetworkContext:Received 9eda82af632e2587d31fcd06d5fb0bfb1df47c4a8383e6a998f26c7c4906a86b result {} from <dffml.df.memory.MemoryOrchestratorContext object at 0x7f240588c040>
DEBUG:dffml.MemoryOrchestratorContext:ctx.outstanding: 0
https://github.com/pdxjohnny/testaaaa {}
9eda82af632e2587d31fcd06d5fb0bfb1df47c4a8383e6a998f26c7c4906a86b {}

Want to understand why we are not waiting for the contexts to complete which were added
in above diff and logs.
- Fallback plan is to call both from a function in a separate overlay until it's working
  this will just call run_custom via a helper function for both README and CONTRIBUTING
  overlays.
  - Going to write this first, then contributing new file tutorial
  - Then tutorial on alice shouldi contribute with overlay addition via installed to entrypoint
  - Then test with ability to add overlays via CLI as one offs
  - Final bit of each tutorial is to add to this fallback overlay
- If we still have time before 8 AM then we'll try to debug
alice: please: contribute: recommended community standards: readme: Scope PR title and body definitions
- 1cf1d73
Tried with dffml.run(), it worked right away. Going with this.
- 1bf5e4c
  - alice: please: contribute: recommended community standards: readme: Use dffml.subflow_typecast to execute README contribution
- 85d57ad
  - high level: subflow typecast: Basic OpImpCtx helper
- 8c0531e
  - overlay: static overlay application with loading entrypoint dataflow class with overlays applied.
- af4306a
  - alice: please: contribute: recommended community standards: Disable OverlayMetaIssue in default installed set of overlays

How to help Alice contribute more files

This tutorial will help you create a new Open Architecture / Alice
overlay which runs when another flow runs. The upstream flow
in this case is the AlicePleaseContributeRecommendedCommunityStandards
base flow.

Copy readme overlay to new file

$ cp alice/please/contribute/recommended_community_standards/readme.py alice/please/contribute/recommended_community_standards/contribute.py

Rename types, classes, variables, etc.

$ sed -e 's/Readme/Contributing/g' -e 's/README/CONTRIBUTING/g' -e 's/readme/contributing/g' -i alice/please/contribute/recommended_community_standards/contribute.py

diff --git a/entities/alice/entry_points.txt b/entities/alice/entry_points.txt
index 129b2866a1..9e130cb3b2 100644
--- a/entities/alice/entry_points.txt
+++ b/entities/alice/entry_points.txt
@@ -9,6 +9,7 @@ CLI                                            = alice.please.contribute.recomme
 OverlayGit                                     = alice.please.contribute.recommended_community_standards.recommended_community_standards:OverlayGit
 OverlayGitHub                                  = alice.please.contribute.recommended_community_standards.recommended_community_standards:OverlayGitHub
 OverlayREADME                                  = alice.please.contribute.recommended_community_standards.recommended_community_standards:OverlayREADME
+OverlayCONTRIBUTING                            = alice.please.contribute.recommended_community_standards.recommended_community_standards:OverlayCONTRIBUTING
 # OverlayMetaIssue                               = alice.please.contribute.recommended_community_standards.recommended_community_standards:OverlayMetaIssue

 [dffml.overlays.alice.please.contribute.recommended_community_standards.overlay.readme]

dffml.git/entites/alice/entry_points.txt

[dffml.overlays.alice.please.contribute.recommended_community_standards.overlay.contributing]
OverlayGit                                     = alice.please.contribute.recommended_community_standards.recommended_community_standards:OverlayGit
OverlayGitHub                                  = alice.please.contribute.recommended_community_standards.recommended_community_standards:OverlayGitHu

Reinstall for new entrypoints to take effect

$ python -m pip install -e .

Re-run the command and observe results

for pr in $(gh -R https://github.com/$USER/ pr list --json number --jq '.[].number'); do gh -R https://github.com/pdxjohnny/testaaaa pr close "${pr}"; done; (alice please contribute -log debug -repos https://github.com/pdxjohnny/testaaaa -- recommended community standards; gh -R https://github.com/pdxjohnny/testaaaa pr list

Script to test Coach Alice Our Open Source Guide tutorial

#!/usr/bin/env bash
set -x
set -e

# export USER=githubusername
export REPO_URL="https://github.com/$USER/my-new-python-project"

cd $(mktemp -d)

git clone --depth=1 -b alice https://github.com/intel/dffml dffml
cd dffml/entities/alice
python -m venv .venv
. .venv/bin/activate
python -m pip install -U pip setuptools wheel
python -m pip install \
  -e .[dev] \
  -e ../../ \
  -e ../../examples/shouldi/ \
  -e ../../feature/git/ \
  -e ../../operations/innersource/ \
  -e ../../configloader/yaml/

gh repo create -y --private "${REPO_URL}"
git clone "${REPO_URL}"
cd my-new-python-project
echo 'print("Hello World")' > test.py
git add test.py
git commit -sam 'Initial Commit'
git push --set-upstream origin $(git branch --show-current)
cd ..
rm -rf my-new-python-project

cp alice/please/contribute/recommended_community_standards/readme.py alice/please/contribute/recommended_community_standards/code_of_conduct.py

sed -e 's/Readme/CodeOfConduct/g' -e 's/README/CODE_OF_CONDUCT/g' -e 's/readme/code_of_conduct/g' -i alice/please/contribute/recommended_community_standards/code_of_conduct.py

sed -i 's/OverlayREADME .*/&\nOverlayCODE_OF_CONDUCT                         = alice.please.contribute.recommended_community_standards.code_of_conduct:OverlayCODE_OF_CONDUCT/' entry_points.txt

tee -a entry_points.txt << 'EOF'

[dffml.overlays.alice.please.contribute.recommended_community_standards.code_of_conduct]
OverlayGit                                     = alice.please.contribute.recommended_community_standards.recommended_community_standards:OverlayGit
OverlayGitHub                                  = alice.please.contribute.recommended_community_standards.recommended_community_standards:OverlayGitHub
EOF

python -m pip install -e .

alice please contribute -log debug -repos "${REPO_URL}" -- recommended community standards

gh -R "${REPO_URL}" pr list
# 343     Recommended Community Standard: README  alice-contribute-recommended-community-standards-readme OPEN
# 341     Recommended Community Standard: CONTRIBUTING    alice-contribute-recommended-community-standards-contributing   OPEN
# 339     Recommended Community Standard: CODE_OF_CONDUCT alice-contribute-recommended-community-standards-code_of_conduct        OPEN

for pr in $(gh -R "${REPO_URL}" pr list --json number --jq '.[].number');
do
  gh -R "${REPO_URL}" pr close "${pr}"
done

The Alice codebase

$ find alice/please/ -type f | grep -v __init
alice/please/contribute/recommended_community_standards/contributing.py
alice/please/contribute/recommended_community_standards/cli.py
alice/please/contribute/recommended_community_standards/readme.py
alice/please/contribute/recommended_community_standards/meta_issue.py
alice/please/contribute/recommended_community_standards/recommended_community_standards.py

TODOs

Explain how to add more top level Alice CLI comamnds
Explain how to overlay shouldi flows beyond standard DFFML docs.

johnandersen777 · 2022-08-22T16:45:06Z

johnandersen777
Aug 22, 2022
Author

2022-08-22 Engineering Logs

SCITT

1 reply

johnandersen777 Aug 22, 2022
Author

2022-08-22 @pdxjohnny Engineering Logs

SCITT
TODO
- Update with some of the very spotty wording above and try to flush it out with more conceptual meat now that the tone is established / future John has an example to work with.
  - 9aeb7f1
- Update somewhere else in Vol 1 to include from
  - https://github.com/intel/dffml/blob/alice/docs/arch/alice/discussion/0015/reply_0002.md

johnandersen777 · 2022-08-24T23:06:08Z

johnandersen777
Aug 24, 2022
Author

2022-08-24 Engineering Logs

SCITT
- https://mailarchive.ietf.org/arch/msg/scitt/R56CX1LqSgDBRCzZIk3pZnJEV_c/
  - “
    In summary, a NIST Vulnerability Disclosure Report (VDR) is an attestation
    by a software vendor showing that the vendor has checked each component of a
    software product SBOM for vulnerabilities and reports on the details of any
    vulnerabilities reported by a NIST NVD search. The VDR is a living document
    which the software vendor updates as needed when new vulnerabilities have
    been discovered and reported. A VDR is published whenever a software vendor
    issues a new or updated SBOM, including initial product release, making it
    available online, all the time, to all customers of the product described in
    the VDR. This gives software consumers that ability to answer the question
    "What is the vulnerability status of my software product from Vendor V, as
    of NOW?".”
  - From VEX to VDR? Lets dive in more next week

2 replies

sedihglow Aug 25, 2022

2022-08-24 @sedihglow Engineering Logs

Alice
- Ran through contributing setup on local PC
  - https://github.com/intel/dffml/blob/alice/entities/alice/CONTRIBUTING.rst#cloning-the-repo
alice please build if needed and run /path/to/repo
- Try two different repos, mainly focused on C
  - https://github.com/sedihglow/rpi4
  - https://github.com/sedihglow/red_black_tree

$ sudo update-alternatives: using /usr/bin/python3.9 to provide /usr/local/bin/python (python) in auto mode
$ sudo apt-get update && sudo apt-get install -y tmux python3.9 python3-pip python3.9-venv python3.9-dev build-essential

johnandersen777 Aug 29, 2022
Author

https://datatracker.ietf.org/doc/draft-ietf-rats-architecture/

johnandersen777 · 2022-08-29T15:01:13Z

johnandersen777
Aug 29, 2022
Author

Policy
- ABC’s of Conformity Assessment
  - https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.2000-01.pdf
  - This might be helpful later when we write docs for / think about how to apply policy (see vol 0 introduction arch diagram)
SCITT
- Zachary Newman shared looking at OpenSSF / SCITT terminology ran into same topics that we did when we brought up using shared underlying protocols and formats in the 2022-07-25 SCITT meeting when talking about RATs style attestation vs SLSA/in-toto/sigstore style.
  - https://mailarchive.ietf.org/arch/msg/scitt/utSOqlCifoorbqUGWNf-wMlBYR4/
    - Dick agrees with Zach's analysis. "I've also been monitoring the OpenSSF Scorecard initiative, which goes beyond sigstore attestation checking to assign a "trust score". Not sure if this has traction, but there is a lot of activity on github. https://github.com/ossf/scorecard/blob/main/README.md#basic-usage OpenSSF does NOT appear to be following/implementing NIST C-SCRM recommendations and standards for Executive Order 14028 and consumer software labeling and other attestation recommendations; https://www.nist.gov/document/software-supply-chain-security-guidance-under-executive-order-eo-14028-section-4e" [Dick Brooks]
- Commit message to charter
  - The Endor POC by @OR13 was exemplary because there was a low amount of abstraction / extra information / steps introduced for the learner to understand the sequence of data transformations involved. It makes clear the contents of the serialization of choice (DIDs + VCs in Endor's case) and how that varies across the steps. The POC provided immediate value on the mailing list in a way that examples which introduce more abstraction layers are unable to do as quickly.
    
    We apply our recent learning from this success by adding to the charter the production of a similar example which in this patch we call "file-based", but we could change that to a more descriptive term if there is one. Having an example similar to the learning methodology presented via Endor would accelerate the pace at which developers up and down the software stack and in different programming languages would be able to adopt SCITT. This is due to the low level of abstraction introduced by it's file and shell based implementation. Files and shell commands translate easy into other languages where they can be slowly swapped out from initial fork/exec and equivalents to language code.
    
    The SCITT community could potentially provide documentation on how the fork/exec style implementation could be transformed into the HTTP server implementation. Due to the generic nature of SCITT and the many touchpoints various software systems will likely have with it in the future. It is important for us to consider as a part of our threat model the effect cohesive example documentation has on the correctness of downstream implementations. Providing cohesive examples where we start with the basics (file-based), moving to an example environment implementers are likely to be working in (HTTP-based), and finally explaining how we went from the basic to the complex would give a robust view of what SCITT should look like to implementers and provide them with a clear path to a hopefully correct implementation.
    
    More cohesive documentation will reduce the number of security vulnerabilities we see in our communities code. Code which is fundamentally about security in nature. This modification to the charter seeks to act on recent learnings around example code experienced within the SCITT community itself and seeks to contribute to the development of our threat model as we think about SCITT's lifecycle and rollout.
- For this reason I propose we
- where they will be creating the future of SCITT's robust, actively maintained solutions.
  - https://mailarchive.ietf.org/arch/msg/scitt/Hz9BSiIN7JHAgsZL6MuDHK4p7P8/
  - https://github.com/OR13/endor
    - This is learning methodology goldmine.
- Increase level of focus and depth to final milestone about documentation / example implementations ietf-scitt/charter#21
- https://mailarchive.ietf.org/arch/msg/scitt/B9cwkueu3gdQ7lBKkhILcFLD0E4/
RATS
- https://datatracker.ietf.org/doc/draft-ietf-rats-architecture/
SBOM
- We [DFFML community] intend to use the "living" SBOM VDR capabilities to facilitate the breathing of life into our living threat models. This will allow us to facilitate vulns on architecture.
  - https://spdx.github.io/spdx-spec/v2.3/
  - https://energycentral.com/c/pip/what-nist-sbom-vulnerability-disclosure-report-vdr
  - The recommendation by NIST to provide software consumers with a NIST VDR is gaining traction as a best practice. The latest version of the SPDX SBOM standard, version 2.3, includes provisions (K.1.9) enabling a software vendor to associate a specific SBOM document for a software product with its online NIST VDR attestation for that product, which is linked within the SBOM. The link refers to a “living” SBOM VDR document that is updated by a software vendor, whenever new vulnerabilities are reported. Having this “always updated NIST VDR” available enables software consumers to answer the question “What is the vulnerability status of my software product from Vendor V, as of NOW?”, providing consumers with on-going, up-to-date visibility into the risks that may be present in an installed software product, as new vulnerabilities (CVE's) are being reported/released.
    
    As stated previously, NIST did not prescribe a format for a NIST VDR attestation, but guidance is provided on what a VDR includes. Reliable Energy Analytics (REA) has produced an open-source “interpretation” of what a NIST VDR contains in order to meet EO 14028, which is available here in an XML Schema format with samples provided in XML and JSON (https://raw.githubusercontent.com/rjb4standards/REA-Products/master/SBOMVDR_JSON/VDR_118.json) formats.

1 reply

johnandersen777 · 2022-08-30T16:46:19Z

johnandersen777
Aug 30, 2022
Author

2022-08-30 Engineering Logs

3 replies

johnandersen777 Aug 31, 2022
Author

https://github.com/opensbom-generator/spdx-sbom-generator

johnandersen777 Aug 31, 2022
Author

https://huggingface.co/spaces/huggingface/diffuse-the-rest

johnandersen777 · 2024-07-12T14:03:01Z

johnandersen777
Jul 12, 2024
Author

https://github.com/denosaurs/deno_python

1 reply

johnandersen777 Jul 12, 2024
Author

LICENSE ubolonton/denopy#4

johnandersen777 · 2024-07-12T18:07:10Z

johnandersen777
Jul 12, 2024
Author

All the major pieces are there

1 reply

johnandersen777 Jul 12, 2024
Author

2024-07-12 World Domination Council

SCITT: All the major pieces are there
tell me what a registration policy is
profile
- need extensibility point for policy
What are our next steps?
Hardware BOM is a huge problem
AJ authoring paper
- https://csrc.nist.gov/Events/2024/enhancing-security-of-devices-and-components
- How to do embargoed vuln sharing
Attestation services talking SCITT
SARIF output metaschema
- Support for SARIF-based validation results usnistgov/metaschema-java#373
https://transparency.dev/summit
- A.J. Let's get implementation
A.J. to chair COSE ActivityPub stuff
- Build off ROLI and CBOR and COSE and COAP and ActivityPub
  - Smallest payload
  - Deltas
https://datatracker.ietf.org/doc/rfc8322/
IETF CORAL
- As long as we can namspace / actor to urns on same IP, for decentralized k8s
- Think RDF but CBOR avoid everything the SPICE people hate about JSON-LD for RDF in VC.
/acc/

Workstream: Hardware Attested Build Environments slsa-framework/slsa#975
- chkimes/image-attestation@bb2033c#diff-f7657169a0123564242ee2e8ed946c773f89897910eb0b3054f1eaa2c87f3711R21
  - swtpm measure PCR values put in SCITT
  - IMA
  - Hiberation
    - Disk image
  - Current client server terminology
    - Server provides quote
    - Client verifies
in-toto policy working group
- https://docs.google.com/document/d/1fcrpj5tD2j3ugmW3-IapaOy69JYGN6PvKBfMa62W53I/edit#heading=h.ab6c414763s0
  - Ramping
  - Verificatoin smmary
  - Backwards compatibility
VCS folds into L4 policy engine
- https://github.com/gittuf/gittuf
  - fine grained access control
/acc/

johnandersen777 · 2024-07-13T15:23:05Z

johnandersen777
Jul 13, 2024
Author

docs: warn about user_metadata used in RLS supabase/supabase#21561

0 replies

johnandersen777 · 2024-07-15T17:37:26Z

johnandersen777
Jul 15, 2024
Author

0 replies

johnandersen777 · 2024-07-16T17:13:06Z

johnandersen777
Jul 16, 2024
Author

https://github.com/bullfrogsec/bullfrog?tab=readme-ov-file

0 replies

johnandersen777 · 2024-07-18T20:45:53Z

johnandersen777
Jul 18, 2024
Author

cose blob for bdiffs
pyexecl llm query

2 replies

johnandersen777 Jul 24, 2024
Author

https://github.com/meta-llama/llama-agentic-system

johnandersen777 Jul 25, 2024
Author

Scitt ietf 120

AJ made a comment reenforcing our policy engine work as being needed within extension docs (effectively) thank you AJ!

johnandersen777 · 2024-07-29T08:15:42Z

johnandersen777
Jul 29, 2024
Author

https://kwasm.sh

2 replies

johnandersen777 Jul 31, 2024
Author

https://dicedb-docs.netlify.app/tutorials/realtime-leaderboard/

johnandersen777 Aug 2, 2024
Author

policy_engine_cwt_rebase_webhooks_in_config failure on config model v… johnandersen777/scitt-api-emulator#8

johnandersen777 · 2024-08-02T16:48:30Z

johnandersen777
Aug 2, 2024
Author

2024-08-02 Engineering Logs

1 reply

johnandersen777 Aug 2, 2024
Author

2024-08-02 @pdxjohnny Engineering Logs

https://github.com/DiceDB/dice
https://github.com/meilisearch/meilisearch
- Use for retrival with agenic rag
https://docs.tea.xyz/tea-white-paper/white-paper#staking-to-secure-the-software-supply-chain
- 🛤️🛤️🛤️🛤️🛤️🛤️🛤️

johnandersen777 · 2024-08-05T00:21:52Z

johnandersen777
Aug 5, 2024
Author

2024-08-04 Engineering Logs

3 replies

johnandersen777 Aug 5, 2024
Author

2024-08-04 @pdxjohnny Engineering Logs

TODO
- AV stream to inventory

johnandersen777 Aug 5, 2024
Author

https://github.com/comfyanonymous/ComfyUI?tab=readme-ov-file#intel-gpus

johnandersen777 Aug 6, 2024
Author

CONTRIBUTING

Commit: https://github.com/intel/dffml/blob/573172cced0fe28ec45d0a85693837b44d4cb8b6/CONTRIBUTING.md#abstract

Pull-Request: CONTRIBUTING: Amalgamize #1660

WARNING: SECURITY

THIS DOCUMENT IS USED AS AN INPUT PROMPT TO LARGE LANGUAGE MODELS

THIS DOCUMENT IS COMBINDED WITH THE PROJECT'S THREATS.md TO DETERMINE TRUSTED CONTRIBUTOR AND COMPUTING BASECLEAR FOR TAKE OFF.

TODO LINK TO IMAGE OR DOC OR SECTION WITH WHITEBOARD SKETCH

Abstract

This document outlines best practices for poly-repo maintainers and contributors, detailing strategic plans and principles for federated code repositories. It emphasizes continuous trust evaluation, automated CI/CD workflows, and federated transparency receipts to ensure alignment with community-agreed values and strategic objectives. The document also addresses developer and maintainer expectations, federation triggers, and the integration of automated checks into CI/CD processes.

Trust boundaries are evaluated continuously, and misalignments trigger retraining of AI models and potential blocking of data events.
Developers and maintainers must document issues, federate forges, and adhere to CI/CD best practices for decentralized governance.
Federation triggers and automated workflows ensure optimal notification and alignment with community values and strategic plans.

Conditions that may result in a lack of federation include:

Misalignment with Strategic Plans: Entities that do not align with the project's strategic values, plans, and principles can be blocked from federating new data events.
Detection of Malicious Intent: Entities suspected of malicious activities or failing to respect shared resources and time may be excluded from federation.
Lack of Contact: When there is no contact with an entity, attempts at federation may be blocked to ensure security and integrity.

Towards More Optimal Communication

Generic best practices for poly-repo maintainers and contributors. 👩🌟🛠️

This doc details the DFFML upstream default strategic plans and principles for entites
our code repository federates with (when your Version Control Software (VCS) has enabled trust
boundry based federation with known users and contributors. Your VCS will come with a default
policy, you may have set one, your org, additional policies are context dependent. When
N entities auto federate version controlled data in alignment with these best practices
the ultimate concequence of repeated detected missalignment to any of these or overlayed
strategic plans and principles.

These measures are in place to maintain trust, security, and alignment with community goals.

When no contact with an entity: Blocking all attempts at federation of new data events directly or when detected in the bill of materials graph analysis of data events from old. This is our trigger for retraining the models which is why it's AI Looped In CI/CD Execution. Overlays may define additional consequencies. Overlays in use get added to all data event BOMs and TCB evaluation is done continuously and retroactivly invalidated if we learn nodes in a graph are not aligned with our running context's strategic plans, principles, and values.
Following these recommendations for communication results in a record of the actions taken
for each train of thought where data has been shared between entities. The authors of this
document have made a best effort attempt to completely capture the these strategic values, plans,
and principles. However, trust evaluation is an ever changing game and context applicable overlays
and future updates to this doc will be done continuously. It should be viewed as a
lessons learned doc and suggested interaction patterns
context and the trust levels of the data within those contexts and the trust level in the
policy evaluation / analysis executed and the

TODO Document the anti-skynet part which is also the part which will auto help filter out people with malicious intent (the on the scam phone call with grandma defense idea)

TODO grep: entities repsecting each others time and compute resource alocation to shared validation. Green energy savings of federated packets received. Value chain alignment analysis
on each commit / packet / new data event: https://docs.tea.xyz/tea-white-paper/white-paper#staking-to-secure-the-software-supply-chain.

AND ACTIONS MAPPED TO COMMUNITY AGREED BOUNDRIES AND TIME COMMITMENT EXPECTATIONS TO RECIEVE SUPPORT FOR REUSE.

This is our default settings for how Alice will interact with an open source community. CI/CD is how we role model the behavior (since she's an AI agent executing policy engine workflows in GitHub Actions format). we expect for and will be defining overlays. It effectivly defines the projects default win criteria: Successful contributions aligned with the purpose of the project. If a patchset aligns it's ready to go. CI/CD for values alignment.

This document outlines requirements and their stringency levels using different examples.
It defines actions taken by maintainers. Timelines for progression and example architypes for varying levels of maintainer / contributor involvement / engagement.

docs: arch: Inventory #1207
arch: Isolation levels #1252
docs: arch: 2nd and 3rd party plugins #1061
df: overlay: Implement middleware/RBAC chains of ordered applications of overlays #1400
util: testing: manifest: shim: Initial commit #1273
manifest: shim: Add JSON schema validation #1300
stream of consiousness: policy engine: step components: 1st party: MetaSchema based evaluation #1657

dffml/docs/arch/alice/discussion/0036/reply_0067.md

Lines 14 to 25 in f8377d0

    
               - Upstream 
        
                 - Past 
        
               - Overlay 
        
                 - Present 
        
               - Orchestration 
        
                 - Takes us to the future 
        
           - The Open Architecture (aka Alice) is 
        
             - A proxy for domain specific representations of architecture 
        
             - The metadata to help us understand each domain specific representation of architecture is called the manifest metadata 
        
             - The term manifest is used to describe a domain specific representation of architecture 
        
             - Each node in the an Open Architecture graph is a manifest 
        
             - The top level document aka Alice aka the Open Architecture itself is a manifest

Best Practices

CI/CD and GitOps facilitate.

TODO Explain strategic plans and principles which the requirements in this doc relate to.

Policy engine workflow execution on issue ops for debug via execution of Agent defined as workflow jobs.
- Capture outputs of this and use it to build the review system as federated transparency receipts for ORSA resolvable artifacts. Were RBAC is defined by policy of which former statement's subject URN is later policy statements transparent receipt URN (relying party output: the workload being given the clear for take off: aka workflow for analysis of BOM and BOM. Manifest of Manifests )

Developers

This section covers call outs on specific behavior or expectations or boundries for developers and.

Document issues in your forge
Federate forges
contributing: github: Videos and frame by frames on how to click around github optimally #1658
contributing: What makes an issue ready for a drive by contribution #1659
- Some way we can see if a complete plan for execution (dataflow / workflow hypothesis and analysis via policy engine)
https://github.com/pdxjohnny/dotfiles/blob/8d9850f85314a9f5c30f5bb7b8e47ba3857357be/forge-install.sh#L1-L584

Architecture Design Records

First have agents write specs as first step, then iterate on them and push the plans to orphan branches on forks, use git lfs for big files.
https://www.rfc-editor.org/pubprocess/
https://www.rfc-editor.org/rfc/rfc8729.html
- Explains governance
Main docs for authors: https://authors.ietf.org/
https://www.ietf.org/how/ids/

Trigger Federation

By interacting with the forge, this will trigger the other entities AI to notify them optimally (TODO find notes on flow state priority interupts only)

git show cfae0f85e:.github/workflows/alice_async_comms.yml | tee .github/workflows/alice_async_comms.yml
gh act run -P ubuntu-latest=ubuntu:22-04 -j posts -W .github/workflows/alice_async_comms.yml

CI/CD

This section covers the integration status of automated checks into the processes.

orphan branches and other best practices to facilitate CI
Eventually SCITT as Version Control Software type stuff aka to facilitate decentralized goverance
- Tutorial Feedback: Architecting Alice: Stream of Consciousness #1415
- https://github.com/builtree/builtree/blob/main/governance/STEERING-COMMITTEE.md

https://thenewstack.io/what-are-the-next-steps-for-feature-flags and the following image sum up the motivations behind the Entity Analysis Trinity, ThreatOps, and VCS+CI/CD a la scitt-community/scitt-api-emulator#27 (comment)

Maintainers

This section covers call outs on specific behavior or expectations or boundries for different maintainers. This defines the threat model / trust boundries and their soul based auth ("Soul of the Software") strategic principle alignment report (a federated ORAS context addressed manifest of manifests which is the "clear for take off").

Boundries: I will federate changes from you and forward changes to others I am federating with if the analysis of the graph of thoughts leading to the send of the new data event is within acceptable risk tolerence. If we notice actions in the network unaligned to the defaults outlined in this document we we will begin enforcing these boundries. Additional boundries can be overlayed

docs: examples: innersource: Org health and issue prioritization #1287

Virtual branch - shared context dependent trains of thought within a poly-repo environment. If the overlays of an entity currently federating with. The N+1 federation new data event is always determined by a KERI duplicity detection protected channel. The tcb for trust evaluation is also party of relying party inputs.

https://github.com/pdxHijohnny/dotfiles/issues/1

Codebase Housekeeping

TODO

THREATS.md

dffml/docs/contributing/codebase.rst

Lines 1 to 183 in f8377d0

    
           Codebase Layout And Notes 
        
           ========================= 
        
           .. contents:: 
        
           Plugins 
        
           ------- 
        
           DFFML is plugin based. This means that the source code for the main 
        
           package ``dffml``, is separate from the source code for many of the things you 
        
           might want to use in conjunction with it. For example, if you wanted to use the 
        
           machine learning models based on scikit, you'd install ``dffml-model-scikit``. 
        
           If you wanted to use machine learning models based on TensorFlow, you'd install 
        
           ``dffml-model-tensorflow``. The source code for all *Official* plugins is within 
        
           the same Git repo (https://github.com/intel/dffml). 
        
           A *Official* plugin is any plugin maintained within the main Git repo. 
        
           This means users only have to install what they need. TensorFlow is several 
        
           hundred megabytes, not everyone wants that, or needs that to get machine 
        
           learning models that perform accurately on their problem. 
        
           All plugins have their base class that they derive from in the main package, 
        
           which is located in the ``dffml`` directory at the root of the git repo. 
        
           The plugin packages are located within their respective directories at the root 
        
           of the git repo. For example, source base classes are in ``dffml/source/`` and 
        
           source plugin packages are in ``source/``. 
        
           Adding A New Plugin 
        
           +++++++++++++++++++ 
        
           To add a new *Official* plugin to DFFML. You need to first create the plugin in 
        
           the appropriate directory. Then add it to the lists of core plugins. 
        
           .. warning:: 
        
               The release process is automated. You should **NOT** upload the package to 
        
               PyPi! Someone from Intel has to be the one to do that for *Official* 
        
               plugins. 
        
           For *Official* plugins, the name given to create should be in the form of 
        
           ``dffml-{PLUGIN_TYPE}-{NAME}``. 
        
           .. code-block:: console 
        
               $ cd model/ 
        
               $ dffml service dev create model dffml-model-someframework 
        
               $ mv dffml-model-someframework someframework 
        
           Now that we've created the plugin, we need to add it to a few lists 
        
           - Add the plugin to ``CORE_PLUGINS`` list in ``dffml/plugins.py`` 
        
           - Update ``.github/workflows/testing.yml`` 
        
             - Add the path to the plugin to the ``jobs.test.strategy.matrix.plugin`` list. 
        
             - Open an issue to have the ``PYPI_{PLUGIN_TYPE}_{NAME}`` token added under 
        
               ``jobs.steps[-1].run``. 
        
               - Sample issue format: ``pypi: Add token for dffml-{PLUGIN_TYPE}-{NAME} to 
        
                 testing.yml`` 
        
           Double Context Entry Pattern 
        
           ---------------------------- 
        
           All classes in DFFML follow a double asynchronous context entry pattern. This is 
        
           ideal for usages such as creating a connection pool, then using a connection. Be 
        
           that with a database, client HTTP sessions, etc. 
        
           .. code-block:: python 
        
               import asyncio 
        
               from dffml.record import Record 
        
               from dffml.source.csv import CSVSource, CSVSourceConfig 
        
               async def main(): 
        
                   # One 
        
                   async with CSVSource( 
        
                       CSVSourceConfig( 
        
                           filename="test.csv", 
        
                           allowempty=True, 
        
                           readwrite=True, 
        
                       ) 
        
                   ) as source: 
        
                       # Two 
        
                       async with source() as sctx: 
        
                           # Punch 
        
                           await sctx.update(Record("0", data={ 
        
                               "features": { 
        
                                   "first_column": 42, 
        
                                   "second_column": 1776, 
        
                               } 
        
                           })) 
        
               asyncio.run(main()) 
        
           Config 
        
           ------ 
        
           Much of the DFFML codebase is dedicated to transforming configuration structures 
        
           between their incoming form to a ``dict`` which can be used to determine what 
        
           plugin needs to be loaded, and what the arguments for the configuration class of 
        
           that plugin are. 
        
           For example: 
        
           .. code-block:: yaml 
        
               model: 
        
                 plugin: tfdnnc 
        
                 config: 
        
                   epochs: 400 
        
                   steps: 4000 
        
                   classifications: 
        
                   - '0' 
        
                   - '1' 
        
                   predict: 
        
                     dtype: int 
        
                     length: 1 
        
                     name: maintained 
        
                   features: 
        
                   - dtype: int 
        
                     length: 10 
        
                     name: authors 
        
                   - dtype: int 
        
                     length: 10 
        
                     name: commits 
        
                   - dtype: int 
        
                     length: 10 
        
                     name: work 
        
           Here, ``plugin`` is the ``...Arg`` class which signifies the plugin to load. 
        
           ``config`` is the ``...Config`` class as a dict for that plugin. 
        
           The command line equivalent for the model is... 
        
           .. code-block:: console 
        
               $ dffml ... \ 
        
                   -model tfdnnc \ 
        
                   -model-epochs 400 \ 
        
                   -model-steps 4000 \ 
        
                   -model-classifications 0 1 \ 
        
                   -model-predict maintained:str:1 \ 
        
                   -model-features \ 
        
                     authors:int:10 \ 
        
                     commits:int:10 \ 
        
                     work:int:10 \ 
        
           setup.py 
        
           -------- 
        
           There are various ``setup.py`` files throughout the codebase, one for the main 
        
           package, one for each plugin, and one in ``skel/``. There are also 
        
           ``setup_common.py`` files. 
        
           Notes on Various Subsystems 
        
           --------------------------- 
        
           DFFML is comprised of various subsystems. The following are some notes 
        
           that might be helpful when working on each of them. 
        
           Working on ``skel/`` 
        
           ++++++++++++++++++++ 
        
           The packages in ``skel/`` are used to create new DFFML packages. 
        
           For example, to create a new package containing operations we run the following. 
        
           .. code-block:: console 
        
               $ dffml service dev create operations dffml-operations-feedface 
        
           If you want to work on any of the packages in ``skel/``, you'll need to run the 
        
           ``skel link`` command first fromt he ``dev`` service. This will symlink required 
        
           files in from ``common/`` so that testing will work. 
        
           .. code-block:: console 
        
               $ dffml service dev skel link

Naming conventions
- Dockerfile
- CI/CD Workflows
- Issues
- Pull Requests

CONTRIBUTING TO CONTRIBUTING

This section defines our values and touches on their contections to strategic plans
and principles in breif. Please modify those in their source of truth sections above
unless the additional context within the values section all community members agree
is required for communication to be well spent time.

Measuring Alignment

TODO Value chain analysis to determine if proposed changes to this document and content thereof have proven aligned with the soul this project's software (aka this doc and THREATS.md: https://intel.github.io/dffml/main/contributing/gsoc/rubric.html)

This rubric is was developed during the 2020 GSoC DFFML proposal review period. Reviewing
proposals is very challenging, we have lots of applicants that do great work and
needed to find a way to quantify their contributions. We use this rubric to do
so.

Alignment Rubric

Alignment Rubric Explanation

Explains why each field was chosen for the rubric. Some fields are subjective,
others are objective. Each mentor will fill out a copy of the rubric for the
subjective portion of the rubric. One mentor will count up relevant statistics
to fill out the objective portion of the rubric. Once all proposals have been
reviewed, the mentors rubrics will be averaged for each proposal. And the final
score will be calculated for each proposal by tallying up all the points in the
averaged subjective rubrics and the objective rubric. The proposals will then
be ranked from highest to lowest. Final scoring will happen two days before
slot requests are due to give students the most time possible to get more
contributions in, get better at debugging, show self direction, and engage with
the community.

Goals in Proposal

The project proposed should have a clear value to the project's community. A
high score here means if the project is selected (and subsequently completed),
users would clearly benefit from it.

Completeness of Proposal

The proposal should contain enough details for it to be clear that the student
knows how to complete the goals they've outlined. More detail beyond proving
they know the basics of what needs to be done is better. If a student proposed
something but was short on detail, their contributions to the project should be
taken into account. If their proposal lacks detail on implementation of
something similar to what they've already done several times, it's not
necessary that they re-explain in detail in the proposal, their contributions
speak for themselves.

Time Commitment

GSoC expects students to spend 30+ hours a week working on their proposed
project / the project they are contributing to (assuming they've completed
their proposed project). Students vary in skill level. As such, mentors should
take their interactions with students into account as well as previous work a
student has completed to estimate if the student will be able to complete the
outlined proposal in the allotted time. Mentors should also think about students
previous work and how long it seemed like they spent working on previous pull
requests to estimate how long each action item in the proposal will take the
student. Using this estimate, mentors should respond in the rubric with their
thoughts on how many hours a week they think the student will be spending
working on their proposed project to complete it.

Engagement with Community

Student engagement with the community is key to their success, the success of
future GSoC applicants, and the project. We cannot effectively collaborate to
build open source software if we don't talk to each other. The best outcome of
GSoC is not only the completion of the student's project, but the student
becoming a mentor for other students or new community members. A high score
here shows active participation in the community and that the students actions
show them trending towards becoming a mentor / maintainer in this organization
or another someday.

Mentor Vote

Every mentor should vote for one proposal which is their personal favorite.
Each mentor has different aspects about the project (DFFML) that are important
to them (maintainers of different subsystems, etc.). Since the mentors will be
advising students, it's important that they believe in the projects they are
guiding. This is how they signal what they believe is important or cool.

Self Directed

While it's always okay to ask what to do, it's better to look at the list of
things to do and pick one. Open source projects put a lot of effort into
project management and organization of work items so that everyone knows what
the open bugs are on the project and what the planned enhancements are.
Everything is done in public because anything is accessible to anyone on the
internet and it's important that we all stay on the same page to effectively
collaborate. Self directed students follow contribution guidelines and
communicate their intentions to work on issues within the posted issue rather
than other channels of communication which don't allow others to see the status
of what's being worked on. This keeps the community in sync so there is no
duplication of work. In some projects (like DFFML) issues are labeled with the
estimated amount of work they will take to complete. Students who take on
larger issues will spend more time working on the problem on their own and less
time with a mentor stepping them through what to do. This is a measure of the
students resourcefulness and tenacity for solving difficult problems.

Debugging

The act of working on any problem, programming in particular, is an exercise in
figuring out what should be done, what has been done, and what's the current
problem that is keeping what has been done from being what should be done. This
loop of fail, fix, repeat, is debugging. Students should be actively testing
their contributions. When those tests fail they need to use critical thinking
to determine if what they've done is working or not. If it's not working they
need to figure out what the next step would be in fixing it, and try that. If
they can't figure it out, then they should ask for help. A student that asks
for help any time there is an issue without trying to figure out what the issue
is before asking for help is not putting in enough effort. In addition, tests
have to pass in the continuous integration (CI) environment. If the test
doesn't pass in CI, then the student should be performing the debugging process
looking at what's wrong in the CI environment. They should not say that they
don't know what is wrong because it works for them. Feigned ignorance of
failing tests does not excuse their failure.

Contributions

Some open source projects (like DFFML) label issues with the estimated amount
of work they will take to complete. Larger contributions, or those associated
with larger issues, receive more points than smaller contributions. The goal
here is to reward students who have put in more effort leading up to GSoC.
They've worked hard to learn about the project they wrote a proposal for,
and have worked hard to make that project better.

Alice: Abstracted Lifecycle Instantiatation of Contributor Efectivo

Upstream: Rolling Alice: Volume 0: Introduction and Context

To think about how we think about editing this document and how it should be
written, imagine an open source developer named Alice. Phrase things here so that
she could ramp up on the project given no previous context, assumputions about
known or practiced development methodologies, strategic principles, plans, and
values. Use this document as if it was the reference with top / override priority
when she is actively working within the context of this project.

Online cloning cuts our iteration time
- Artificial Life Is Coming Eventually
- Data flows are the parallel exploration of trains of thought (nested graphs)
- Natural selection and evolution
  - Tree of life
  - Parallel exploration of nested graphs
- Automated synchronization of system state across distinct timelines (distinct roots)
  - Enables the resolution of system state post haste, post state, and post date
    - See fuzzy finding later in this doc: find "join disparate roots"
    - This is effectively out of order execution at a higher level of abstraction, in the aggregate, so as to bring the aggregate set of agents involved to an equilibrium state
- We are building the thought communication protocol, to communicate thought is to learn
  - If we can describe any architecture, any problem space, we can describe any thought
  - To describe a thought most completely, one must know how to best communicate with that entity
  - That entity, that agent, is a moving target for communication at it's optimal rate of learning.
  - It's past is relevant in determining it's future as it's past determines what will resonate best with it in terms of forming conceptual linkages.
    - Past doesn't have to be memory, data and compute are the same in our architecture
    - Hardwired responses get encoded the same way, it's all the signal, the probability
- When Alice goes through the looking glass she'll take us with her in sprit, and come back to communicate to us how best to proceed, in every way.
  - The less (more?) rambling way of putting this would be, we need our AI to be a true to us extension of ourselves, or of our ad-hoc formed groups, they need to be true to those strategic principles we've communicated to the machine. If we can trust their transparency (estimates/forecasts and provenance on that) about their ability to stay aligned to those principles, then we can accurately assess operating risk and it's conformance to our threat model or any threat model the execution of the job fits within.
  - This means we can trust our AI to not influence us in the wrong ways.
  - This means we can trust it to influence us in the right ways, the ways we want to influence ourselves, or our software development lifecycle.
    - This assessment of the level of trust fundamentally comes from our analysis of our analysis of our software development lifecycle, our Entity Analysis Trinity.

Entity Analysis Trinity

We leverage the Entity Analysis Trinity to help us bridge the gap between
our technical activities and processes and the conceptual model we are
following as we analyze the softare / system / entity over it's lifecycle.

Terminology

Open Architecture
- Universal Blueprint
- Standard architecture we use to describe anything. Provides the ability to use / reference domain specific architectures as needed to define architecture of whole.
- https://github.com/intel/dffml/blob/main/docs/arch/0009-Open-Architecture.rst
Think
- Come up with new data flows and system context input
Thoughts
- Data Flows and system context input pairs (these two plus orchestration config we get the whole system context)
Downstream Validation
- Running validation on all dependent packages to check for API breakages or regressions in the ecosystem

Expectations

Alice is going to be held to very high standards. We should expect this list to grow for a long time (years). This list of expectations may at times contain fragments which need to be worked out more and are only fragment so the ideas don't get forgotten.

Alice should be able to work on any project as a remote developer
- She should be able to make changes to projects following the branch by abstraction methodology
- When she works on a github issue she'll comment what commands she tries and what files she modifies with diffs
Alice will maintain a system which allows her to respond to asynchronous messages
- Likely a datastore with the ability to listen for changes
- Changes would be additions of messages from different sources (email, chat, etc.)
Alice should be able to accept a meeting, join it, and talk to you
- If Alice notices conversation getting off topic, she could interject to ask how it relates, and then update references in docs to that effect.
- You should be able to have a conversation about a universal blueprint and she should be able to go act on it.
- She should be able to analyze any codebase you have access to live and build and walk you through architecture diagrams
- Alice build me a linux distro with these versions of these applications deploy it in a VM in QEMU, show me the screen while it's booting. Then give me control of it via this meeting. ... Okay now snapshot and deploy to XYZ CSP.
  - She should figure out how to validate that she has a working linux distro by overlaying discovered tests with intergration tests such as boot check via qemu serial.
- Alice, spin up ABC helm charts and visualize the cluster (viewing in an AR headset)
- Alice, let's talk about the automating classification web app included in the example.
  - Alice, give us an overview of the threats on our database, deploy the prod backup to a new environment. Attempt to exploit known threats and come up with new ones for the next 2 weeks. Submit a report and presentation with your findings. Begin work on issues found as you find them.
We should be able to see Alice think and understand her trains of thought
- If Alice is presenting and she estimates thinking of the correct solution will take longer than a reasonable time her next word is expected by to keep regular conversational cadence, she should either offer to brainstorm, work through it and wait until it makes sense to respond, maybe there are situations where the output is related to saving someone's life, then maybe she interupts as soon as she's done thinking. Provided she didn't detect that the train of thought which was being spoken about by others was not of higher prioritiy than her own (with regards to lifesaving metrics).

Alice's Understanding of Software Engineering

We'll teach Alice what she needs to know about software engineering though our InnerSource series. She'll follow the best practices outlined there. She'll understand a codebase's health in part using InnerSource metric collectors.

Alice will see problems and look for solutions. Problems are gaps between the present system capabilities and desired system capabilities or interpretations of outputs of strategic plans which are unfavorable by the strategic decision maker or the prioritizer.

========================= IN PROGRESS NOTES DUMPS BEYOND THIS =====================

About

Data Flow Facilitator for Machine Learning (DFFML) makes it easy to generate
datasets, train and use machine learning models, and integrate machine learning
into new or existing applications. It provides APIs for dataset generation,
storage, and model definition.

Models handle implementations of machine learning algorithms.
Likely wrapping code from a popular machine learning framework.
Sources handle the storage of datasets, saving and loading them from files,
databases, remote APIs, etc.
DataFlows are directed graphs used to generate a dataset, as well as modify
existing datasets. They can also be used to do non-machine learning tasks, you
could use them to build a web app for instance.

You'll find the existing implementations of all of these on their respective
:ref:plugins pages. DFFML has a plugin based architecture, which allows us to
include some sources, models, and operations as a part of the main package,
dffml, and other functionality in more specific packages.

Mission Statement

As we all know the Machine Learning space has a lot of tools and libraries for
creating pipelines to train, test & deploy models, and dealing with these many
different APIs can be cumbersome.

Our project aims to make this process a breeze by introducing interoperability
under a modular and easily extensible API. DFFML’s plugin-based architecture makes
it a swiss army knife of ML research & MLOps.

We heavily rely on DataFlows, which are basically directed graphs. We are also
working on a WebUI to make dataflows completely a drag’n drop experience.
Currently, all of our functionalities are accessible through Python API, CLI,
and HTTP APIs.

We broadly have two types of audience here, one is Citizen Data Scientists and
ML researchers, who’d probably use the WebUI to experiment and design models.
MLOps people will deploy models and set up data processing pipelines via the
HTTP/CLI/Python APIs.

What is key objective of DataFlows

Our objective is to provide an environment where users can describe what they
want done, without needing to know how it gets done.
By separating the what from the how, each implementation of how a piece of
work gets done doesn’t need to know anything about the implementations of how
other pieces of work get done.
This approach allows domain experts to implement the work within their
domain, without needing to know anything about how the work of other domain
experts is done.
In such an environment, individual pieces of work implemented by respective
domain experts can be knit together to accomplish arbitrary tasks.
By taking this approach of separating the what from the how, we’ve created an
environment where anyone can describe what they want to get done. How it gets
done is handled by executing implementations provided by domain experts.

Why is this objective important?

Our existing software development model requires that experts in various
domains maintain implementations of software which must be actively kept
compatible with each other.
By adopting this new model, we maximize workplace efficiency by compatibility
between domain expert work being done at an orchestration level.

What industry challenges do DataFlows address / solve?

We address and provide a solution for the challenge of how software written by
experts in different domains can be used together without integration
overhead.
We make it easy for domain experts to share their work.
We make it easy for users to reuse the work of domain experts which reduces
duplication of effort across the industry.
In this ecosystem, where work is easily shared and integrated, software
development becomes more efficient and produces software of higher quality.

Key Takeaways

Flow-based programming makes it easy to share and integrate existing work.
The software development lifecycle is streamlined by implementing this model.

Why DFFML

You want a "just bring your data" approach to machine learning.
- No need to write code if you don't want to, use popular machine learning
  libraries via the command line, high level Python abstraction, or HTTP API.
You want to do machine learning on a new problem that you don't have a dataset
for, so you need to generate it.
- Directed Graph Execution lets you write code that runs concurrently with
  managed locking. Making the feature engineering iteration process very fast.

Architecture

This is a high level overview of how DFFML works.

.. TODO Autogenerate image during build

graph TD

subgraph DataFlow[Dataset Generation]
  df[Directed Graph Execution]
  generate_features[Generate Feature Data]
  single[Single Record]
  all[Whole DataSet]

  df --> generate_features
  generate_features --> single
  generate_features --> all
end

subgraph ml[Machine Learning]
  train[Model Training]
  accuracy[Model Accuracy Assessment]
  predict[Prediction Using Trained Model]
end

subgraph sources[Dataset Storage]
  source[Dataset Storage Abstraction]
  JSON
  CSV
  MySQL

  source --> JSON
  source --> CSV
  source --> MySQL
end

all --> train
all --> accuracy
single --> predict

generate_features --> source
predict --> source

.. image:: https://dffml.github.io/dffml-pre-image-removal/master/_images/arch.svg
:alt: Architecture Diagram

Machine Learning

Python was chosen because of the machine learning community’s preference towards
it. In addition to the data flow side of DFFML, there is a machine learning
focused side. It provides a standardized way to defining, training, and using
models. It also allows for wrapping existing models so as to expose them via the
standardized API. Models can then be integrated into data flows as operations.
This enables trivial layering of models to create complex features. See
:ref:plugin_models for existing models and usage.

Data Flows - Directed Graph Execution

The idea behind this project is to provide a way to link together various new
or existing pieces of code and run them via an orchestration engine that
forwards the data between them all. Similar a microservice architecture but with
the orchestration being preformed according to a directed graph. This offers
greater flexibility in that interaction between services can easily be modified
without changing code, only the graph (known as the dataflow).

This is an example of the dataflow for a meta static analysis tool for Python,
shouldi. We take the package name (package) and feed it through operations,
which are just functions (but could be anything, some SaaS web API endpoint for
instance). All the data generated by running these operations is query-able,
allowing us to structure the output in whatever way is most fitting for our
application.

.. image:: https://dffml.github.io/dffml-pre-image-removal/master/_images/shouldi-dataflow.svg
:alt: DataFlow for shouldi tool

Consistent API

DFFML decouples the interface through which the flow is accessed from the flow
itself. For instance, data flows can be run via the library, HTTP API, CLI, or
any communication channel (next targets are Slack and IRC). Data flows are also
asynchronous in nature, allowing them to be used to build any event driven
application (Chat, IoT data, etc.). The way in which operations are defined and
executed by the orchestrator will let us take existing API endpoints and code in
other languages and combine them into one cohesive workflow. The architecture
itself is programming language agnostic, the first implementation has been
written in Python.

Plugins

We take a community driven approach to content. Architecture is plugin based,
which means anyone can swap out any piece by writing their own plugin and
publishing it to the Python Package Index. This means that developers can
publish operations and machine learning models that work out of the box with
everything else maintained as a part of the core repository and with other
developers models and operations. :doc:tutorials/index show how to create your
own plugins.

Users

The following is a list of organizations and projects using DFFML. Please let us
know if you are using DFFML and we'll add you to the list. If you want help
using DFFML, see the :doc:contact page.

Intel
- Security analysis of Open Source Software dependencies.

Philosophy

DFFML is an event driven directed graph execution architecture tightly coupled
with the typical machine learning workflow. The core concept is that all
software can be looked at as a set of operations that occur in response to
asynchronous events. Directed graphs are used to specify which operations should
be run in response to which events. Every event has data associated with it,
therefore we refer to the directed graph as a DataFlow.

The project as it exists now is a Python library which provides data set
generation via DataFlows, dataset storage, as well as model training, testing,
and inference. Users can leverage DataFlows to do feature engineering, to create
new datasets and modify or add to existing datasets. They then train models,
assess their accuracy and use them to make predictions via various deployment
methods.

DFFML has a plugin based architecture. Every model, data source, operation, etc.
is a plugin. We maintain a set of official plugins which wrap various machine
learning frameworks such as Daal4Py, TensorFlow, Scikit Learn, etc. By wrapping
frameworks in a standard API we simplify usage and make it easy for developers
to switch from one underlying framework to another.

Conceptually, DFFML is not just the Python implementation it is today. It’s a
programming language agnostic architecture centered around the concept of
DataFlows and the decoupling of definition from implementation. One goal of the
project closely associated with this is to have an orchestrator (Alice <https://github.com/intel/dffml/tree/main/entities/alice/>_) capable of
deploying and knitting together new or existing services without the need for
those services to know anything about each other. This could be thought of as a
level of abstraction beyond serverless architecture, which is where we're hoping
to take the project.

Team

We have an awesome team working on the project. We hold weekly meetings
and have a mailing list and chat! If you want to get involved, ask questions, or
get help getting started, see :doc:contact.

We participated in Google Summer of Code 2019 under the Python Software
Foundation. A big thanks to our students, Yash and Sudharsana!

:doc:GSoC 2019 Student Contributions <contributing/gsoc/2019>

We are currently participating in Google Summer of Code 2020 under the Python
Software Foundation. Big thanks to Aghin, Himanshu, and Saksham!

:doc:GSoC 2020 Student Contributions <contributing/gsoc/2020>

Thank you to everyone who's contributed to DFFML!!!

Abdallah Bashir
Aghin Shah Alin
Arvindh Kumar Chandran
Aryan Gupta
Byambaa
Constanza Heath
Dentigg
Dmitry Poliuha
Govindarajan Panneerselvam
Hashim
Himanshu Tripathi
iamandeepsandhu
Jan Keromnes
John Andersen
Joseph Kato
Justin Moore
Naeem Khoshnevis
NeerajBhadani
NMNDV
Pankaj Patil
pradeepbhadani
purnimapatel
raghav-ys
Saksham Arora
Sanket Saurav
shivam singh
Sudhanshu kumar
Sudharsana K J L
Taksh Kamlesh
Theo
us
Vaibhav Mehra
Yash Lamba
Yash Varshney

.. Generated with git log --format=format:'%an' | sort | uniq
You'll want to filter out duplicates if you re-generate this

johnandersen777 · 2024-08-10T15:16:34Z

johnandersen777
Aug 10, 2024
Author

2024-08-10 Engineering Logs

2 replies

johnandersen777 Aug 10, 2024
Author

Rolling Alice: Volume 0: Chapter 11: Entering Wonderland

Moved to: https://github.com/intel/dffml/blob/4d054ce4a626145fac642122e46cd23aeed60abc/docs/tutorials/rolling_alice/0000_architecting_alice/0011_entering_wonderland.md#rolling-alice-volume-0-chapter-11-entering-wonderland

The Meaning of Meaningless Riddles

The Merits of Insanity: With enough Chaos comes a closer to objective view of reality. The meaning of meaningless riddles once reaching full insanity becomes only that they are meaningless. They cease to be riddles.

We drop them via policy on federation and re-evaluate past thoughts. Tthe prioritizer as supply chain analysis of thoughts shows the data seeding those riddles does not in fact come from trusted TCBs.

Good_Insane = prioritizer.value_chain_analysis(Good_Insane + Good_Insane)

These TCBs might have gone the bad way of insane.

Bad_Insane = prioritizer.value_chain_analysis(Good_Insane + Bad_Insane)

What We Do When 1+1 No Longer Equals 2

Upon detection of a faulty root of trust we must trigger re-evaluation of tainted trains of thought
Roots of trust may begin producing data ungrounded from any shared reality in error through no fault of their own. We must attempt to reconstruct affected trains of thought via GUAC and the like to identify deviations and recorded and hypothetical risk tolerance violations.

continuous reevaluation of old trains of thought
Discovery phase (execution of scientific process) yields results with higher trust / TCB assurances
re-train
Kick off CI/CD/MLOps
The dionysian discovery method: shift left to high entropy (insanity) to re-check theories within previously unknown contexts given new data events which triggered re-eval which came from higher trust boundries

TODO
- Index federated data by entities which affected it so we can purge our cache as needed

https://www.youtube.com/watch?v=n10aZDfjsoc - Alice Through the Looking Glass (2016) - Mad Hatter wakes up: Mad Hatter wakes up after Alice realizes and understands him and his promise.

johnandersen777 Aug 10, 2024
Author

2024-08-10 @pdxjohnny Engineering Logs

https://github.com/teableio/teable
- Forge
https://github.com/MahmoudAshraf97/whisper-diarization
- Writing the Wave

johnandersen777 · 2024-08-11T13:10:47Z

johnandersen777
Aug 11, 2024
Author

2024-08-11 Engineering Logs

1 reply

johnandersen777 Aug 11, 2024
Author

2024-08-11 @pdxjohnny Engineering Logs

These are the known and unintended simplifications so far that KERI, ACDC, and CESR have brought to the world:

KERI’s novel pre-rotating mechanism (intended) means private keys are quantum-attack resistant via a clever trick (spin-off).

KERI has cut out intermediaries (intended) from the equation in binding between the controller, private, and public keys. The purely cryptographic nature of these bindings simplifies them: they are all cryptographically strong and end-to-end verifiable (spin-off).

KERI is omnipresent and trust-spanning (intended). Simplification: No blockchain silos with a need to build and maintain interoperability forever (spin-off).

CESR (needed) has ended the streaming encoding wars between CBOR, JSON, and MSGPACK; it handles them all (spin-off).

CESR has round-robin composability between text and binary (intended). This eliminates the need to prove the consistency and authenticity of readable legal digital documents when sent over the Internet (spin-off).

In ACDCs, the system guides you to sign everything* at rest (KERI’s KEL and TEL) and in motion (CESR). You don’t have to ask yourself whether this is signed off. It is always, anywhere, by anyone or anything.

Most verifiable credentialing systems sign at rest. ACDCs bind the issuance to the key state at the time of issuance. They are bound or sealed at rest to the key state, and that key state is signed. The ACDCs themselves are not signed but have Seals anchored in TELS which in turn have Seals anchored in KELS.

KERI solves the problem of having to reissue all issuances (it’s necessary) every time you rotate keys.

The all-encompassing non-repudiability and duplicity evidence of operations that seal, bind, or anchor (synonyms in KERI) actions to the controller of the current signing key of a KEL simplifies life substantially: IT personnel can’t “get away” anymore with acting maliciously. Until today, in systems without KERI, a system administrator can easily obfuscate his actions. Any external party cannot do a risk assessment. Often, only weeks or months later, malicious actors with root user rights can be demasked. KERI makes life easier.

If you now consider that KERI is, just “maybe,” a bit complicated while delivering these overwhelming spin-off simplifications, then my work is all but done.

Conclusion
KERI is “AND and AND”:

It’s predictable, therefore at the maximum complicated, instead of complex) AND Although KERI’s features and code and interrelations between components aren’t reducible as such (next article), its complicatedness can be reduced by black-boxing KERI AND KERI’s killer feature is that it simplifies the Identity space by combining planned and unforeseen characteristics. Cliff hanger Critics only perceive KERI in the narrow definition of complicated. We can and will further reduce KERI’s complication by working on 5W’s: Wallet, Witness, Watchers, Web, and Wizard.

Would we ever reduce complications by throwing essential parts out?'

And if we could throw out non-essential parts, would we have complied with our own minimal-sufficient-solution rule in the first place?

Twice, the answer is ‘no.’ For those still puzzled and thinking, ‘How can you reduce complications if you don’t throw anything out?’ ->

That’s the topic of another article.

https://github.com/trustoverip/tswg-cesr-specification

johnandersen777 · 2024-08-12T17:02:19Z

johnandersen777
Aug 12, 2024
Author

https://github.com/geekan/MetaGPT/tree/main/examples/di

0 replies

johnandersen777 · 2024-08-13T14:08:17Z

johnandersen777
Aug 13, 2024
Author

https://mailarchive.ietf.org/arch/msg/scitt/mt9DVhe4leEEZSPXIRGNf9D41pE/

1 reply

johnandersen777 Aug 13, 2024
Author

https://github.com/wez/wezterm
- docs: arch: Inventory #1207

johnandersen777 · 2024-08-14T15:23:30Z

johnandersen777
Aug 14, 2024
Author

https://github.com/electric-sql/electric/tree/main/examples/web-wa-sqlite

0 replies

johnandersen777 · 2024-08-16T14:21:08Z

johnandersen777
Aug 16, 2024
Author

0 replies

Alice Engineering Comms 🪬 #1406

johnandersen777 Jul 18, 2022

Engineering Log Process

Replies: 691 comments · 523 replies

johnandersen777 Jul 18, 2022 Author

2022-07-18 Engineering Logs

johnandersen777 Jul 18, 2022 Author

2022-07-18 @pdxjohnny Engineering Logs

johnandersen777 Jul 19, 2022 Author

2022-07-19 Engineering Logs

johnandersen777 Mar 7, 2023 Author

2022-07-19 @pdxjohnny Engineering Logs

johnandersen777 Jul 20, 2022 Author

2022-07-20 Engineering Logs

johnandersen777 Jul 20, 2022 Author

2022-07-20 @pdxjohnny Engineering Logs

johnandersen777 Jul 20, 2022 Author

2022-07-20 Identifying Security Threats WG

johnandersen777 Jul 21, 2022 Author

2022-07-21 Engineering Logs

johnandersen777 Jul 23, 2022 Author

2022-07-23

johnandersen777 Jul 24, 2022 Author

2022-07-28 Alice Intelligence/Open Architecture Working Group Initial Meeting

johnandersen777 Jul 25, 2022 Author

2022-07-25 Engineering Logs

johnandersen777 Jul 25, 2022 Author

2022-07-25 @pdxjohnny Engineering Logs

johnandersen777 Jul 25, 2022 Author

2022-07-25 Supply Chain Integrity, Transparency and Trust (SCITT)

johnandersen777 Jul 26, 2022 Author

2022-07-26 Engineering Logs

johnandersen777 Jul 26, 2022 Author

2022-07-26 @pdxjohnny Engineering Logs

johnandersen777 Jul 27, 2022 Author

2022-07-27 Engineering Logs

johnandersen777 Jul 28, 2022 Author

2022-07-27 @pdxjohnny Engineering Logs

Refactoring and Thinking About Locking of Repos for Contributions

johnandersen777 Jul 28, 2022 Author

2022-07-28 Engineering Logs

johnandersen777 Jul 28, 2022 Author

2022-07-28 @pdxjohnny Engineering Logs

43

44

45

johnandersen777 Jul 29, 2022 Author

2022-07-29 Engineering Logs

johnandersen777 Jul 29, 2022 Author

2022-07-29 @pdxjohnny Engineering Logs

Exploring a Helper Around Run DataFlow run_custom

Refactor into README and CONTRIBUTING Overlays

How to help Alice contribute more files

Script to test Coach Alice Our Open Source Guide tutorial

TODOs

johnandersen777 Aug 22, 2022 Author

2022-08-22 Engineering Logs

johnandersen777 Aug 22, 2022 Author

2022-08-22 @pdxjohnny Engineering Logs

johnandersen777 Aug 24, 2022 Author

2022-08-24 Engineering Logs

sedihglow Aug 25, 2022

2022-08-24 @sedihglow Engineering Logs

johnandersen777 Aug 29, 2022 Author

johnandersen777 Aug 29, 2022 Author

johnandersen777 Aug 29, 2022 Author

2022-08-29 @pdxjohnny Engineering Logs

johnandersen777
Jul 18, 2022

Replies: 691 comments 523 replies

johnandersen777
Jul 18, 2022
Author

johnandersen777 Jul 18, 2022
Author

johnandersen777
Jul 19, 2022
Author

johnandersen777 Mar 7, 2023
Author

johnandersen777
Jul 20, 2022
Author

johnandersen777 Jul 20, 2022
Author

johnandersen777 Jul 20, 2022
Author

johnandersen777
Jul 21, 2022
Author

johnandersen777
Jul 23, 2022
Author

johnandersen777
Jul 24, 2022
Author

johnandersen777
Jul 25, 2022
Author

johnandersen777 Jul 25, 2022
Author

johnandersen777 Jul 25, 2022
Author

johnandersen777
Jul 26, 2022
Author

johnandersen777 Jul 26, 2022
Author

johnandersen777
Jul 27, 2022
Author

johnandersen777 Jul 28, 2022
Author

johnandersen777
Jul 28, 2022
Author

johnandersen777 Jul 28, 2022
Author

johnandersen777
Jul 29, 2022
Author

johnandersen777 Jul 29, 2022
Author

johnandersen777
Aug 22, 2022
Author

johnandersen777 Aug 22, 2022
Author

johnandersen777
Aug 24, 2022
Author

johnandersen777 Aug 29, 2022
Author

johnandersen777
Aug 29, 2022
Author

johnandersen777 Aug 29, 2022
Author