Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update sphinx youtube plugin #1619

Merged
merged 2 commits into from
May 4, 2023
Merged

update sphinx youtube plugin #1619

merged 2 commits into from
May 4, 2023

Conversation

pingsutw
Copy link
Member

@pingsutw pingsutw commented May 4, 2023

Use a maintained package for the sphinx youtube plugin

Signed-off-by: Kevin Su <[email protected]>
@codecov
Copy link

codecov bot commented May 4, 2023

Codecov Report

Merging #1619 (375383d) into master (baf0b89) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #1619   +/-   ##
=======================================
  Coverage   70.10%   70.10%           
=======================================
  Files         328      328           
  Lines       30156    30156           
  Branches     5452     5452           
=======================================
  Hits        21141    21141           
+ Misses       8485     8473   -12     
- Partials      530      542   +12     

see 8 files with indirect coverage changes

@pingsutw pingsutw merged commit c03635d into master May 4, 2023
eapolinario pushed a commit that referenced this pull request May 16, 2023
* update sphinx youtube plugin

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

---------

Signed-off-by: Kevin Su <[email protected]>
eapolinario added a commit that referenced this pull request May 17, 2023
* Data subsystem (#1526)

Signed-off-by: Ketan Umare <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>

* Pluck retry from flytekit and into sagemaker (#1411)

* Remove retry from flytekit's setup.py and regenerate requirements

Signed-off-by: Eduardo Apolinario <[email protected]>

* Add to sagemaker

Signed-off-by: Eduardo Apolinario <[email protected]>

* Remove retry from sagemaker plugin requirements file

Signed-off-by: Eduardo Apolinario <[email protected]>

* Restore doc-requirements.txt

Signed-off-by: eduardo apolinario <[email protected]>

* Fix bad merge

Signed-off-by: eduardo apolinario <[email protected]>

---------

Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: eduardo apolinario <[email protected]>
Co-authored-by: Eduardo Apolinario <[email protected]>

* Update the pypi wait (#1554)

Signed-off-by: Yee Hing Tong <[email protected]>

* Stream Directories and Files using Flyte (#1512)

Signed-off-by: Ketan Umare <[email protected]>
Signed-off-by: Niels Bantilan <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>

* Make `FlyteFile` compatible with `Annotated[..., HashMethod]` (#1544)

* fix: Make FlyteFile compatible with Annotated[..., HashMethod]

See issue #3424

Signed-off-by: Adrian Rumpold <[email protected]>

* tests: Add test case for FlyteFile with HashMethod annotation

Issue: #3424
Signed-off-by: Adrian Rumpold <[email protected]>

* fix: Use typing_extensions.Annotated for py3.8 compatibility

Issue: #3424
Signed-off-by: Adrian Rumpold <[email protected]>

* fix: Use `get_args` and `get_origin` from typing_extensions for py3.8 compatibility

Issue: #3424
Signed-off-by: Adrian Rumpold <[email protected]>

* fix(tests): Use fixture for local dummy file

Signed-off-by: Adrian Rumpold <[email protected]>

---------

Signed-off-by: Adrian Rumpold <[email protected]>

* move FlyteSchema deprecation warning to initialization method (#1558)

* move FlyteSchema deprecation warning to initialization method

Signed-off-by: Niels Bantilan <[email protected]>

* update

Signed-off-by: Niels Bantilan <[email protected]>

---------

Signed-off-by: Niels Bantilan <[email protected]>

* add pod_template and pod_template_name arguments for ContainerTask (#1515)

* add pod_template and pod_template_name arguments for ContainerTask

Signed-off-by: Felix Ruess <[email protected]>

* factor out _serialize_pod_spec into separate util function

Signed-off-by: Felix Ruess <[email protected]>

* model file changes, couple other changes

Signed-off-by: Yee Hing Tong <[email protected]>

* minor cleanup

Signed-off-by: Felix Ruess <[email protected]>

* add unit test for container_task pod_template

Signed-off-by: Felix Ruess <[email protected]>

* bump min version of flyteidl to 1.3.12

for pod template data config support

Signed-off-by: Felix Ruess <[email protected]>

* require flyteidl==1.3.12 in doc-requirements.txt

Signed-off-by: Felix Ruess <[email protected]>

---------

Signed-off-by: Felix Ruess <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>
Co-authored-by: Yee Hing Tong <[email protected]>

* Pass locally defined scopes to RemoteClientConfigStore (#1553)

Signed-off-by: franco-bocci <[email protected]>

* TypeTransformer for TensorFlow model (#1562)

* TypeTransformer for TensorFlow model

Signed-off-by: Samhita Alla <[email protected]>

* clean up

Signed-off-by: Samhita Alla <[email protected]>

* clean up

Signed-off-by: Samhita Alla <[email protected]>

* fix lint

Signed-off-by: Samhita Alla <[email protected]>

---------

Signed-off-by: Samhita Alla <[email protected]>

* Remove fsspec flytekit plugin from main Dockerfile (#1561)

Signed-off-by: Yee Hing Tong <[email protected]>

* Device auth flow / Headless auth (#1552)

* Device auth flow

Signed-off-by: Ketan Umare <[email protected]>

* Device AuthFlow is now available in flytekit

Signed-off-by: Ketan Umare <[email protected]>

* unit tests

Signed-off-by: Ketan Umare <[email protected]>

* test added

Signed-off-by: Ketan Umare <[email protected]>

* updated

Signed-off-by: Ketan Umare <[email protected]>

* Fixed test

Signed-off-by: Ketan Umare <[email protected]>

* Fixed unit test

Signed-off-by: Ketan Umare <[email protected]>

* Fixed lint errors

Signed-off-by: Ketan Umare <[email protected]>

---------

Signed-off-by: Ketan Umare <[email protected]>

* url encode secret in client credentials flow (#1566)

* url encode secret first

Signed-off-by: Yee Hing Tong <[email protected]>

* nit

Signed-off-by: Yee Hing Tong <[email protected]>

---------

Signed-off-by: Yee Hing Tong <[email protected]>

* Python run multiple files (#1559)

* wip

Signed-off-by: Kevin Su <[email protected]>

* wip

Signed-off-by: Kevin Su <[email protected]>

* test

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* lint

Signed-off-by: Kevin Su <[email protected]>

* update comment

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

---------

Signed-off-by: Kevin Su <[email protected]>

* General Partial support in flytekit and multi-list support in flytekit (#1556)

Signed-off-by: Ketan Umare <[email protected]>

* fix: Silence keyring warnings by changing to debug (#1568)

Co-authored-by: ggydush-fn <[email protected]>

* Support GCP secrets (#1571)

Signed-off-by: Yee Hing Tong <[email protected]>

* Automatically remove unused import (#1574)

* Automatically remove unused import

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

---------

Signed-off-by: Kevin Su <[email protected]>

* Disallow partial lists in map tasks (#1577)

* Disallow partial lists in map tasks

Signed-off-by: eduardo apolinario <[email protected]>

* Lint

Signed-off-by: eduardo apolinario <[email protected]>

---------

Signed-off-by: eduardo apolinario <[email protected]>
Co-authored-by: eduardo apolinario <[email protected]>

* Remove duplicate reporting logic (#1578)

Signed-off-by: Yee Hing Tong <[email protected]>

* [Core feature] Convert List[Any] to a single pickle file (#1535)

* Convert List[Any] to a single pickle file

* remove redundant code

* keep batchSize only if type contain flytePickle

* fix error

* add batch support to translate_inputs_to_literals

* add ci test

Signed-off-by: Yicheng-Lu-llll <[email protected]>

* improve comment

Signed-off-by: Yicheng-Lu-llll <[email protected]>

* add more ci test

Signed-off-by: Yicheng-Lu-llll <[email protected]>

* improve

Signed-off-by: Yicheng-Lu-llll <[email protected]>

* handle HashMethod case

Signed-off-by: Yicheng-Lu-llll <[email protected]>

* improve format

Signed-off-by: Yicheng-Lu-llll <[email protected]>

* improve

Signed-off-by: Yicheng-Lu-llll <[email protected]>

* improve

Signed-off-by: Yicheng-Lu-llll <[email protected]>

* improve

Signed-off-by: Yicheng-Lu-llll <[email protected]>

* add test_is_batchable

Signed-off-by: Yicheng-Lu-llll <[email protected]>

* Add BatchSize

Signed-off-by: Kevin Su <[email protected]>

* test_batch_size

Signed-off-by: Kevin Su <[email protected]>

---------

Signed-off-by: Yicheng-Lu-llll <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Co-authored-by: root <[email protected]>
Co-authored-by: Kevin Su <[email protected]>

* Improve authoring structure documentation (#1572)

Signed-off-by: Samhita Alla <[email protected]>

* Unify sqlalchemy Dockerfiles (#1585)

* Unify sqlalchemy Dockerfiles

Signed-off-by: eduardo apolinario <[email protected]>

* Simplifyte sqlalchemy Dockerfile

Signed-off-by: eduardo apolinario <[email protected]>

* Set pythonpath to /root

Signed-off-by: eduardo apolinario <[email protected]>

---------

Signed-off-by: eduardo apolinario <[email protected]>
Co-authored-by: eduardo apolinario <[email protected]>

* Pyflyte build imageSpec (#1555)

* wip

Signed-off-by: Kevin Su <[email protected]>

* wip

Signed-off-by: Kevin Su <[email protected]>

* wip

Signed-off-by: Kevin Su <[email protected]>

* wip

Signed-off-by: Kevin Su <[email protected]>

* wip

Signed-off-by: Kevin Su <[email protected]>

* wip

Signed-off-by: Kevin Su <[email protected]>

* wip

Signed-off-by: Kevin Su <[email protected]>

* wip

Signed-off-by: Kevin Su <[email protected]>

* wip

Signed-off-by: Kevin Su <[email protected]>

* wip

Signed-off-by: Kevin Su <[email protected]>

* wip

Signed-off-by: Kevin Su <[email protected]>

* wip

Signed-off-by: Kevin Su <[email protected]>

* pyflyte build

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* wip

Signed-off-by: Kevin Su <[email protected]>

* wip

Signed-off-by: Kevin Su <[email protected]>

* wip

Signed-off-by: Kevin Su <[email protected]>

* Support serialize and package

Signed-off-by: Kevin Su <[email protected]>

* more tests

Signed-off-by: Kevin Su <[email protected]>

* wip

Signed-off-by: Kevin Su <[email protected]>

* wip

Signed-off-by: Kevin Su <[email protected]>

* wip

Signed-off-by: Kevin Su <[email protected]>

* move to plugin

Signed-off-by: Kevin Su <[email protected]>

* test

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* test

Signed-off-by: Kevin Su <[email protected]>

* test

Signed-off-by: Kevin Su <[email protected]>

* test

Signed-off-by: Kevin Su <[email protected]>

* test

Signed-off-by: Kevin Su <[email protected]>

* wip

Signed-off-by: Kevin Su <[email protected]>

* wip

Signed-off-by: Kevin Su <[email protected]>

* wip

Signed-off-by: Kevin Su <[email protected]>

* wip

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* fixed tested

Signed-off-by: Kevin Su <[email protected]>

* fixed tested

Signed-off-by: Kevin Su <[email protected]>

* more tests

Signed-off-by: Kevin Su <[email protected]>

* Add support passing yaml in pyflyte run

Signed-off-by: Kevin Su <[email protected]>

* lint

Signed-off-by: Kevin Su <[email protected]>

* lint

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

---------

Signed-off-by: Kevin Su <[email protected]>

* Feature/3506: Improve Type Conversion errors, use rich to prettify error messages (#1582)

* improve input type conversion error

Signed-off-by: Niels Bantilan <[email protected]>

* fix lint

Signed-off-by: Niels Bantilan <[email protected]>

* fix

Signed-off-by: Niels Bantilan <[email protected]>

* add tests

Signed-off-by: Niels Bantilan <[email protected]>

* add tests

Signed-off-by: Niels Bantilan <[email protected]>

* add rich

Signed-off-by: Niels Bantilan <[email protected]>

* fix lint

Signed-off-by: Niels Bantilan <[email protected]>

* remove prototyping script, update loggers

Signed-off-by: Niels Bantilan <[email protected]>

* update __init__.py

Signed-off-by: Niels Bantilan <[email protected]>

* update logger

Signed-off-by: Niels Bantilan <[email protected]>

* update logger

Signed-off-by: Niels Bantilan <[email protected]>

* fix GE and pandera tests

Signed-off-by: Niels Bantilan <[email protected]>

* fix lint

Signed-off-by: Niels Bantilan <[email protected]>

---------

Signed-off-by: Niels Bantilan <[email protected]>

* Add support nested FlyteFile in pyflyte run (#1587)

* Add support nested FlyteFile in pyflyte run

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

---------

Signed-off-by: Kevin Su <[email protected]>

* Set cache_regions=True in the case of s3fs (#1592)

* Set cache_regions=True in the case of s3fs

* Linting

Signed-off-by: eduardo apolinario <[email protected]>

* Fix tests

Signed-off-by: eduardo apolinario <[email protected]>

* More linting

Signed-off-by: eduardo apolinario <[email protected]>

---------

Signed-off-by: eduardo apolinario <[email protected]>
Co-authored-by: eduardo apolinario <[email protected]>

* Set default ComSpec when running on Windows (#1595)

* set default ComSpec when running on Windows

Signed-off-by: Kevin Su <[email protected]>

* lint

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

---------

Signed-off-by: Kevin Su <[email protected]>

* Feature: add activate-launchplan command to pyflyte (#1588)

* add activate-launchplan command to pyflyte

Signed-off-by: Artem Petrov <[email protected]>

* fix linting

Signed-off-by: Artem Petrov <[email protected]>

* rework launchplan command

Signed-off-by: Artem Petrov <[email protected]>

* fix linting

Signed-off-by: Artem Petrov <[email protected]>

---------

Signed-off-by: Artem Petrov <[email protected]>

* nit (#1596)

Signed-off-by: Yee Hing Tong <[email protected]>

* Add log streaming to papermill plugin (#1129)

* checkpoint

Signed-off-by: Mike Zhong <[email protected]>

* Experimental implementation works well. Instead of messing with the class, we utilize the interpolation and context to dynamically generate a wrapper around the desired script. In the wrapper, we cd to the ctx.working_directory, export the env variables (handled by an additional method to convert dict to str), and then pass the arguments to the script

Signed-off-by: Mike Zhong <[email protected]>

* fix spacing

Signed-off-by: Mike Zhong <[email protected]>

* remove breakpoint

Signed-off-by: Mike Zhong <[email protected]>

* Output ctx.working_directory as single output

Signed-off-by: Mike Zhong <[email protected]>

* more doc strings

Signed-off-by: Mike Zhong <[email protected]>

* more comments

Signed-off-by: Mike Zhong <[email protected]>

* Added comments

Signed-off-by: Mike Zhong <[email protected]>

* Add tests and test files from other branch

Signed-off-by: Mike Zhong <[email protected]>

* minor fix, have function return the instance rather than create. It seems to get registered when flyte packages your project

Signed-off-by: Mike Zhong <[email protected]>

* fix tests

Signed-off-by: Mike Zhong <[email protected]>

* remove set flags not supported by sh

Signed-off-by: Mike Zhong <[email protected]>

* fix spellcheck lint errors

Signed-off-by: Mike Zhong <[email protected]>

* don't have a windows equivalent test script so bypassing those tests for now

Signed-off-by: Mike Zhong <[email protected]>

* Add typing to make_export_string_from_env_dict params

Signed-off-by: Mike Zhong <[email protected]>

* fixup doc string

Signed-off-by: Mike Zhong <[email protected]>

* Refactored the new behavior out into a new class. Did not change implementation details at all

Signed-off-by: Mike Zhong <[email protected]>

* Address linter issues and up test cov

Signed-off-by: Mike Zhong <[email protected]>

* Skip test on windows, no equivalent script

Signed-off-by: Mike Zhong <[email protected]>

* Run black and isort, address SC2236

Signed-off-by: Mike Zhong <[email protected]>

* Refactored class name to remove _, make utility function require name so multiple uses in a workflow don't break

Signed-off-by: Mike Zhong <[email protected]>

* fix utility function

Signed-off-by: Mike Zhong <[email protected]>

* fix utility function call in tests

Signed-off-by: Mike Zhong <[email protected]>

* Fix isort on test_shell.py

Signed-off-by: Mike Zhong <[email protected]>

* adding logging settings to papermill plugin

Signed-off-by: Calvin Leather <[email protected]>

* set level to info logging

* improved comments/docs

---------

Signed-off-by: Mike Zhong <[email protected]>
Signed-off-by: Calvin Leather <[email protected]>
Co-authored-by: Mike Zhong <[email protected]>
Co-authored-by: Mike Zhong <[email protected]>

* Set lower bound for s3fs (#1598)

* set lower bound for s3fs

Signed-off-by: Kevin Su <[email protected]>

* update doc-requirements.txt

Signed-off-by: Kevin Su <[email protected]>

---------

Signed-off-by: Kevin Su <[email protected]>

* Implement horovod task in mpi plugin (#1575)

* Add horovod task to mpi plugin

Signed-off-by: byhsu <[email protected]>

* Remove unused

Signed-off-by: byhsu <[email protected]>

* add test

Signed-off-by: byhsu <[email protected]>

* inherit from mpifunctiontask

Signed-off-by: byhsu <[email protected]>

* fix format

Signed-off-by: byhsu <[email protected]>

* fix format

Signed-off-by: byhsu <[email protected]>

---------

Signed-off-by: byhsu <[email protected]>
Co-authored-by: byhsu <[email protected]>

* add ImageRender class for Flyte Decks (#1599)

Signed-off-by: esad <[email protected]>

* Bump tensorflow from 2.10.0 to 2.11.1 in /plugins/flytekit-mlflow (#1586)

Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.10.0 to 2.11.1.
- [Release notes](https://github.com/tensorflow/tensorflow/releases)
- [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md)
- [Commits](tensorflow/tensorflow@v2.10.0...v2.11.1)

---
updated-dependencies:
- dependency-name: tensorflow
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Handle batchable lists of length 0 (#1600)

Signed-off-by: eduardo apolinario <[email protected]>
Co-authored-by: eduardo apolinario <[email protected]>

* pyflyte run imperative workflow (#1597)

* pyflyte run imperative workflow

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* lint

Signed-off-by: Kevin Su <[email protected]>

---------

Signed-off-by: Kevin Su <[email protected]>

* Backfill should fill in the right input vars (#1593)

* Backfill fix

- Backfill was using incorrect arguments
- backfill should use the argument that user provides or none at all

Signed-off-by: Ketan Umare <[email protected]>

* Updated code

Signed-off-by: Ketan Umare <[email protected]>

* fixed unit test

Signed-off-by: Ketan Umare <[email protected]>

---------

Signed-off-by: Ketan Umare <[email protected]>

* Pyflyte prettified (#1602)

* Pyflyte prettified

Signed-off-by: Ketan Umare <[email protected]>

* added dependency

Signed-off-by: Ketan Umare <[email protected]>

---------

Signed-off-by: Ketan Umare <[email protected]>

* pyflyte run now supports json/yaml files (#1606)

* pyflyte run now supports json files

Signed-off-by: Ketan Umare <[email protected]>

* added yaml support

Signed-off-by: Ketan Umare <[email protected]>

* fixed parsing

Signed-off-by: Ketan Umare <[email protected]>

* fixed windows test

Signed-off-by: Ketan Umare <[email protected]>

---------

Signed-off-by: Ketan Umare <[email protected]>

* Add is_inside func to ImageSpec (#1601)

* Add is_inside func to ImageSpec

Signed-off-by: Kevin Su <[email protected]>

* test

Signed-off-by: Kevin Su <[email protected]>

* Add is_inside func to ImageSpec

Signed-off-by: Kevin Su <[email protected]>

* Add tests

Signed-off-by: Kevin Su <[email protected]>

* rename

Signed-off-by: Kevin Su <[email protected]>

* rename

Signed-off-by: Kevin Su <[email protected]>

* check execution mode

Signed-off-by: Kevin Su <[email protected]>

---------

Signed-off-by: Kevin Su <[email protected]>

* Measuring the time taken for each component when executing a task (#1581)

* add time measurement framework

Signed-off-by: Yicheng-Lu-llll <[email protected]>

* add time measurement framework

Signed-off-by: Yicheng-Lu-llll <[email protected]>

* nit

Signed-off-by: Yicheng-Lu-llll <[email protected]>

* nit

Signed-off-by: Yicheng-Lu-llll <[email protected]>

* nit

Signed-off-by: Yicheng-Lu-llll <[email protected]>

* nit

Signed-off-by: Yicheng-Lu-llll <[email protected]>

* nit

Signed-off-by: Yicheng-Lu-llll <[email protected]>

---------

Signed-off-by: Yicheng-Lu-llll <[email protected]>

* Skip the image building process if the check for its existence fails (#1614)

* Fixed tests

Signed-off-by: Kevin Su <[email protected]>

* test

Signed-off-by: Kevin Su <[email protected]>

* test

Signed-off-by: Kevin Su <[email protected]>

* test

Signed-off-by: Kevin Su <[email protected]>

* lint

Signed-off-by: Kevin Su <[email protected]>

* wip

Signed-off-by: Kevin Su <[email protected]>

* test

Signed-off-by: Kevin Su <[email protected]>

* test

Signed-off-by: Kevin Su <[email protected]>

* test

Signed-off-by: Kevin Su <[email protected]>

* test

Signed-off-by: Kevin Su <[email protected]>

* test

Signed-off-by: Kevin Su <[email protected]>

* test

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* add lru cache

Signed-off-by: Kevin Su <[email protected]>

* update

Signed-off-by: Kevin Su <[email protected]>

* lint

Signed-off-by: Kevin Su <[email protected]>

---------

Signed-off-by: Kevin Su <[email protected]>

* Enable torch elastic training (torchrun) (#1603)

Signed-off-by: Ketan Umare <[email protected]>
Signed-off-by: Fabio Graetz <[email protected]>

This PR brings [torch elastic training (`torchrun`)](https://pytorch.org/docs/stable/elastic/run.html) to the pytorch plugin:

```python
from flytekitplugins.kfpytorch import Elastic

@task(
    task_config=Elastic(
        replicas=4,
        nproc_per_node=4,
        ...
    ),
    ...
)
def train(...):
    ...
```

flyteorg/flyte#3614

* update sphinx youtube plugin (#1619)

* update sphinx youtube plugin

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

---------

Signed-off-by: Kevin Su <[email protected]>

* Read polar dataframe without copying to local (#1618)

Signed-off-by: Kevin Su <[email protected]>

* Add default spark image (#1616)

* Add default spark image

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* fix tests

Signed-off-by: Kevin Su <[email protected]>

* Fix tests

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

---------

Signed-off-by: Kevin Su <[email protected]>

* Lazy load modules (#1590)

* lazy load module

Signed-off-by: Kevin Su <[email protected]>

* lazy load module

Signed-off-by: Kevin Su <[email protected]>

* import

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* keep structured dataset in flytekit init

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* fixed tess

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* fixed tests

Signed-off-by: Kevin Su <[email protected]>

* fixed tests

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* move import pandas to __init__

Signed-off-by: Kevin Su <[email protected]>

* use lazy import loader instead

Signed-off-by: Kevin Su <[email protected]>

* Fixed tests

Signed-off-by: Kevin Su <[email protected]>

* Fixed tests

Signed-off-by: Kevin Su <[email protected]>

* wip

Signed-off-by: Kevin Su <[email protected]>

* fix tests

Signed-off-by: Kevin Su <[email protected]>

* regular import

Signed-off-by: Kevin Su <[email protected]>

* fixed tests

Signed-off-by: Kevin Su <[email protected]>

* test

Signed-off-by: Kevin Su <[email protected]>

* lint

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* lint

Signed-off-by: Kevin Su <[email protected]>

---------

Signed-off-by: Kevin Su <[email protected]>

* Override node name with a name containing underscore (#1608)

* override node_name with a name containing underscore

Signed-off-by: Kevin Su <[email protected]>

* add comment

Signed-off-by: Kevin Su <[email protected]>

---------

Signed-off-by: Kevin Su <[email protected]>

* External Plugin Service (grpc) (#1524)

External Plugin Service

* Include traceback in errors from admin (#1621)

Signed-off-by: Yee Hing Tong <[email protected]>

* Add support for copying all the files in source root  (#1622)

* Add support for copying all the files in source root

Signed-off-by: Kevin Su <[email protected]>

* Add tests

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

---------

Signed-off-by: Kevin Su <[email protected]>

* fix PipelineModel transformer issue 3648 (#1623)

Signed-off-by: esad <[email protected]>

* update GH workflow for external plugin service (#1624)

Signed-off-by: Kevin Su <[email protected]>

* [Issue-3617] Enables FlyteFiles, FlyteDirectors, and StructuredDatasets inputs in papermill plugin (#1612)

* Add support for using a list as an input for a subworkflow (#1605)

Signed-off-by: Kevin Su <[email protected]>

* Improve task decorator type hints with overload (#1631)

Without the overload, the decorated function does not have the proper type of PythonFunctionTask, leading to spurious type errors when attempting to register the task on a FlyteRemote object

Signed-off-by: Matthew Hoffman <[email protected]>

* add annotation option for serialization (#1615)

* add annotation option for serialization

* add support for guessing type

* formatting

* test both directions

* remove guessing for annotated types

* clarify usage in test

* Update tests/flytekit/unit/core/test_type_engine.py

---------

Co-authored-by: Eli Bixby <[email protected]>
Co-authored-by: Yee Hing Tong <[email protected]>

* Delete removed data persistence classes from docs (#1633)

Signed-off-by: Niels Bantilan <[email protected]>

* Improve workflow decorator type hints with overload (#1635)

Previously, the workflow decorator is hinted as always returning a WorkflowBase, which is not true when _workflow_function is None; similar to #1631, we propose using typing.overload to differentiate the return type of workflow based on the value of _workflow_function

Signed-off-by: Matthew Hoffman <[email protected]>

* Fix mypy errors (#1313)

* wip

Signed-off-by: Kevin Su <[email protected]>

* Fix mypy errors

Signed-off-by: Kevin Su <[email protected]>

* Fix mypy errors

Signed-off-by: Kevin Su <[email protected]>

* Fix tests

Signed-off-by: Kevin Su <[email protected]>

* Fix tests

Signed-off-by: Kevin Su <[email protected]>

* Fix tests

Signed-off-by: Kevin Su <[email protected]>

* wip

Signed-off-by: Kevin Su <[email protected]>

* wip

Signed-off-by: Kevin Su <[email protected]>

* fix tests

Signed-off-by: Kevin Su <[email protected]>

* fix tests

Signed-off-by: Kevin Su <[email protected]>

* fix test

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* Update type

Signed-off-by: Kevin Su <[email protected]>

* Fix tests

Signed-off-by: Kevin Su <[email protected]>

* Fix tests

Signed-off-by: Kevin Su <[email protected]>

* Fix tests

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* update dev-requirements.txt

Signed-off-by: Kevin Su <[email protected]>

* Address comment

Signed-off-by: Kevin Su <[email protected]>

* upgrade torch

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* lint

Signed-off-by: Kevin Su <[email protected]>

---------

Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Co-authored-by: Yee Hing Tong <[email protected]>

* type-protobuf<4

Signed-off-by: eduardo apolinario <[email protected]>

* Allow for python 3.7

Signed-off-by: eduardo apolinario <[email protected]>

* Fix protobuf transformer merge error

* Fix bad merge of s/_task_models/task_models/

* Add maxsize=128 to @lru_cache to cover python 3.7

Signed-off-by: eduardo apolinario <[email protected]>

* Lint

Signed-off-by: eduardo apolinario <[email protected]>

* Regenerate doc-requirements.txt

Signed-off-by: eduardo apolinario <[email protected]>

* Add missing call to lru_cache

Signed-off-by: eduardo apolinario <[email protected]>

* Bump version and regenerate requirements in a few plugins

Signed-off-by: eduardo apolinario <[email protected]>

* Revert "Fix mypy errors (#1313)"

This reverts commit af49155.

Signed-off-by: Eduardo Apolinario <[email protected]>

* Remove docs

Signed-off-by: eduardo apolinario <[email protected]>

* More linting

Signed-off-by: eduardo apolinario <[email protected]>

* Fix type of get_type_for_output_var

Signed-off-by: eduardo apolinario <[email protected]>

* Remove tensorflow test_model

* Replace use of shutil.copytree

Signed-off-by: eduardo apolinario <[email protected]>

* Other use of copytree

Signed-off-by: eduardo apolinario <[email protected]>

* Stringify paths in test_script_mode

Signed-off-by: eduardo apolinario <[email protected]>

* os.path.isfile instead in imagespec

Signed-off-by: eduardo apolinario <[email protected]>

* Remove tests

Signed-off-by: eduardo apolinario <[email protected]>

* More linting

Signed-off-by: eduardo apolinario <[email protected]>

* Remove flaky test

Signed-off-by: eduardo apolinario <[email protected]>

---------

Signed-off-by: Ketan Umare <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: eduardo apolinario <[email protected]>
Signed-off-by: Niels Bantilan <[email protected]>
Signed-off-by: Adrian Rumpold <[email protected]>
Signed-off-by: Felix Ruess <[email protected]>
Signed-off-by: franco-bocci <[email protected]>
Signed-off-by: Samhita Alla <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Yicheng-Lu-llll <[email protected]>
Signed-off-by: Artem Petrov <[email protected]>
Signed-off-by: Mike Zhong <[email protected]>
Signed-off-by: Calvin Leather <[email protected]>
Signed-off-by: byhsu <[email protected]>
Signed-off-by: esad <[email protected]>
Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Matthew Hoffman <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Co-authored-by: Yee Hing Tong <[email protected]>
Co-authored-by: Eduardo Apolinario <[email protected]>
Co-authored-by: Ketan Umare <[email protected]>
Co-authored-by: Adrian Rumpold <[email protected]>
Co-authored-by: Niels Bantilan <[email protected]>
Co-authored-by: Felix Ruess <[email protected]>
Co-authored-by: Franco Bocci <[email protected]>
Co-authored-by: Samhita Alla <[email protected]>
Co-authored-by: Kevin Su <[email protected]>
Co-authored-by: ggydush <[email protected]>
Co-authored-by: ggydush-fn <[email protected]>
Co-authored-by: Yicheng-Lu-llll <[email protected]>
Co-authored-by: root <[email protected]>
Co-authored-by: Artem <[email protected]>
Co-authored-by: Calvin Leather <[email protected]>
Co-authored-by: Mike Zhong <[email protected]>
Co-authored-by: Mike Zhong <[email protected]>
Co-authored-by: ByronHsu <[email protected]>
Co-authored-by: byhsu <[email protected]>
Co-authored-by: peridotml <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Fabio M. Graetz, Ph.D <[email protected]>
Co-authored-by: Matthew Hoffman <[email protected]>
Co-authored-by: Eli Bixby <[email protected]>
Co-authored-by: Eli Bixby <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants