Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding a notebook demostrating video transcript translate with whisper in AutoGen #881

Merged
merged 22 commits into from
Dec 7, 2023

Conversation

chengxuan233
Copy link
Contributor

Why are these changes needed?

This is a Notebook using Whisper and GPT-3.5-turbo with AssistantAgent and UserProxyAgent to recognize and translate the speech sound from a video file and add the timestamp like a subtitle file.
This is part of a project from the course DS 440 at Penn State University.

Related issue number

Checks

@chengxuan233
Copy link
Contributor Author

@microsoft-github-policy-service agree [company="{Penn State University}"]

@BeibinLi
Copy link
Collaborator

BeibinLi commented Dec 5, 2023

Fantastic! I just took a quick look at the notebook. Two minor suggestions

  1. Can we combine the two notebooks together? For instance, we can have two sections within the same notebook.
  2. Can we use an online video instead of a local file, so that other users can try it without changing anything.

@chengxuan233
Copy link
Contributor Author

@microsoft-github-policy-service agree [company="{company="{your company}"] I have sole ownership of intellectual property rights to my Submissions and I am not making Submissions in the course of work for my employer.
@microsoft-github-policy-service agree

@chengxuan233
Copy link
Contributor Author

@microsoft-github-policy-service agree

@qingyun-wu
Copy link
Contributor

Thanks @chengxuan233. Can you do the following:

  1. add a link to this notebook in AutoGen website in this page: https://github.com/microsoft/autogen/blob/main/website/docs/Examples.md
  2. remove the redundant notebook file as suggested by @BeibinLi

@chengxuan233
Copy link
Contributor Author

The path has been added to example.md.
For the online video, it seems the 'FFmpeg' package does not support online file reading. I have added the instructions for downloading the example video locally to the notebook.

@codecov-commenter
Copy link

codecov-commenter commented Dec 7, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (e96213e) 26.58% compared to head (50524b1) 26.58%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #881   +/-   ##
=======================================
  Coverage   26.58%   26.58%           
=======================================
  Files          28       28           
  Lines        3732     3732           
  Branches      847      847           
=======================================
  Hits          992      992           
  Misses       2667     2667           
  Partials       73       73           
Flag Coverage Δ
unittests 26.52% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@qingyun-wu qingyun-wu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks!

@qingyun-wu qingyun-wu enabled auto-merge December 7, 2023 20:06
@qingyun-wu qingyun-wu added this pull request to the merge queue Dec 7, 2023
Merged via the queue into microsoft:main with commit 1b4fb8f Dec 7, 2023
16 checks passed
rlam3 pushed a commit to rlam3/autogen that referenced this pull request Dec 19, 2023
…r in AutoGen (microsoft#881)

* add agentchat_video_transcript_translate.ipynb

* finish the agentchat_video_transcript_translate.ipynb file notebook

* modify the recognize_transcript_from_video function

* run the script and add the output to the notebook

* implement the notebook

* add the link to the video clip

* rename the file and add the version requirement of each packages

* add the new notebook path

* add the notebook path to Example.md

* add the new notebook path to the new example.md

* add the instruction of FFmpeg and video download

* Update Examples.md

* Update Examples.md

* Update Examples.md

* Update Examples.md

* Delete notebook/agentchat_video_transcript_translate.ipynb

* Update Examples.md and add the link

---------

Co-authored-by: silver233jpg <[email protected]>
whiskyboy pushed a commit to whiskyboy/autogen that referenced this pull request Apr 17, 2024
…r in AutoGen (microsoft#881)

* add agentchat_video_transcript_translate.ipynb

* finish the agentchat_video_transcript_translate.ipynb file notebook

* modify the recognize_transcript_from_video function

* run the script and add the output to the notebook

* implement the notebook

* add the link to the video clip

* rename the file and add the version requirement of each packages

* add the new notebook path

* add the notebook path to Example.md

* add the new notebook path to the new example.md

* add the instruction of FFmpeg and video download

* Update Examples.md

* Update Examples.md

* Update Examples.md

* Update Examples.md

* Delete notebook/agentchat_video_transcript_translate.ipynb

* Update Examples.md and add the link

---------

Co-authored-by: silver233jpg <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants