Valay/real time cards sidecar-updates #69
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
valayDave
changed the title
valayDave
force-pushed
the
valay/real-time-cards
branch
7 times, most recently
from
c260334 to
c569d22
Compare
- added interface to MetaflowCardComponent for making it REALTIME_UPDATABLE - add realtime component rendering capabiity. - Changed ville's abstraction of CardComponents to CardComponentManager - introduced the `current.card.components` / `current.card['abc'].components` interface - `current.card.components` interface helps access/remove components
- abstraction to handle card creation for decorator (and in future outside code) - Create CardProcessManager class that helps manage the card processes running - Remove `_card_proc` logic from code. - fix card-refresh-bug caused by cardproc instance-method refactor - the `_card_proc` method is an instance method of a card decorator which is passed to componentCollector. This is done so that ComponentCollector can call the method when a refresh is called for an individual card. - Since there is only one copy of the ComponentCollector it created an issue when other cards were trying to call refresh (since ComponentCollector was instantiated with a **single card decorator's** `_card_proc`) - This commit refactored the code to handle multiple cards calling refresh
valayDave
force-pushed
the
valay/real-time-cards
branch
from
c569d22 to
ce2693c
Compare
valayDave
force-pushed
the
valay/real-time-cards-scu
branch
from
fe8c657 to
3bc3465
Compare
valayDave
force-pushed
the
valay/real-time-cards
branch
from
ce2693c to
cdd6fb1
Compare
- cleanup the ui component code and add naive progress-bar
…vior - Markdown and Artifact Component made realtime. - default/blank cards made realtime
…dates - Modified card datastore to accomodate retrieval of runtime data updates - Added a card viewer html file - Created a simple HTTP based card server that will help showcase the realtime cards from querying the server - Card datastore's read and write path retrieval methods now explicitly are given the suffix they retrieve from. We do this because the suffix determines if we are extracting a card or a data update - Added a private method in the `Card` (user-interface) to get the data related to a card.
valayDave
force-pushed
the
valay/real-time-cards
branch
from
cdd6fb1 to
1be4ab0
Compare
valayDave
force-pushed
the
valay/real-time-cards-scu
branch
from
3bc3465 to
9bf5496
Compare
- Created classes that will help dump current.card state - created functions that will help retrieve current.card in a separate subprocess. What is missing : 1. Multi-process update safety 2. Constraints on sharing / not sharing components with main process
- created a locking mechanism to ensure cards are locked to individual subprocesses.
- introduced `LockableCardDict` which helps lock cards per subprocess
- introduced `CardStateManager` to save the state of components/data updates during task execution
## life-cycle of cards
1. Pre-task execution :
- @card decorator instantiates a CardComponentCollector with a LockableCardDict set to `pre_task_startup=True` and empty dictionary for it's `card_component_store`
- This ensures that new CardComponentManagers can be trivially added by the main process.
- Once main process is done adding all cards, we run `finalize` method :
- it will set `pre_task_startup=False` and set post_task_startup=True
- it will instantiate the locks. Won't check if they are already possessed by others since the code is being run in the main process and
task execution has not started yet; (All lock alteration happens once user code starts executing).
- Upon finishing finalize `current.card` will also dump the state of `current.card` for being available to any outside subprocesses.
2. Task Execution:
- Main process :
- Which ever cards the main thread update's will be locked by the main thread.
- The cards which are not accessed / updated by the main thread, can be locked by other processes.
- Locked cards will provide the same type of object back to the user, but the `card_proc` method will be replaced with a
method that will warn the user that the card is locked.
- Subprocesses:
- calls the `get_runtime_card` method to get the `current.card` object
- the method will seek out the `current.card`'s state location and load the state.
- the state of the card specifies what ever task spec's component collector's state
- will be able to call render_runtime and refresh from the cli
- `refresh` will also dump the state of the components to disk.
- [todo] when finished will call `current.card.done()` which will dump the state of the components/data-updates to disk
- [todo] no other processes will also be able to lock "done" cards
- [todo] If a subprocess is gracefully shutdown, then:
- [todo] the card manager will call `current.card.done()` which will avoid writes from other processes
3. Post Task Execution:
- sub processes:
- Should have **ideally** ended and released all locks and ran the finalize method.
- If not then what ever is the last component state dumped in the subprocess will be the components loaded for the card.
- Main process:
- For each card,
- call the render method.
- [todo] If the card was locked by an outside process then seek it's component state and load it.
- if the card
valayDave
force-pushed
the
valay/real-time-cards-scu
branch
from
9bf5496 to
d12cd98
Compare
valayDave
force-pushed
the
valay/real-time-cards
branch
from
8911806 to
cf4cd58
Compare
valayDave
force-pushed
the
valay/real-time-cards
branch
6 times, most recently
from
276088d to
357495b
Compare
valayDave
force-pushed
the
valay/real-time-cards
branch
7 times, most recently
from
068db2f to
1b73af4
Compare
valayDave
force-pushed
the
valay/real-time-cards
branch
from
1b73af4 to
38fea65
Compare
valayDave
force-pushed
the
valay/real-time-cards
branch
5 times, most recently
from
6b19add to
853d640
Compare
valayDave
force-pushed
the
valay/real-time-cards
branch
2 times, most recently
from
12abd7f to
7192efd
Compare
valayDave
force-pushed
the
valay/real-time-cards
branch
from
7192efd to
36613dc
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.