added project structure and necessary docs

Author: swainshashwat

.ipynb_checkpoints/Untitled-checkpoint.ipynb

@@ -0,0 +1,36 @@
+# Flock Project Code of Conduct
+
+As contributors and maintainers of the Flock project, we pledge to create and maintain a welcoming, respectful, and inclusive environment for everyone involved. We value the diversity of backgrounds and experiences and are committed to providing a harassment-free and positive experience for all.
+
+## Our Standards
+
+Examples of behavior that contribute to creating a positive environment include:
+
+- **Being respectful:** Treat all individuals with kindness, empathy, and respect, regardless of their background or identity.
+- **Being inclusive:** Encourage and welcome participation from all individuals, regardless of their background or experience level.
+- **Being considerate:** Be mindful of your words and actions, and avoid any behavior that could be perceived as offensive, discriminatory, or disrespectful.
+- **Being collaborative:** Work together to achieve common goals, and support fellow contributors in their efforts.
+
+Examples of unacceptable behavior include:
+
+- **Harassment:** Any form of harassment, intimidation, or discrimination will not be tolerated.
+- **Unwelcome behavior:** Deliberate exclusion, name-calling, trolling, or insulting/derogatory comments.
+- **Inappropriate content:** Posting or sharing inappropriate or offensive material.
+
+## Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective actions in response to any instances of unacceptable behavior.
+
+Contributors to the Flock project are expected to adhere to these standards both within project spaces and in public spaces when discussing the project or interacting with other community members.
+
+## Reporting
+
+If you experience or witness behavior that violates our Code of Conduct, please promptly report it to [[email protected]](mailto:[email protected]). All reports will be reviewed and investigated, and appropriate action will be taken.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant](https://www.contributor-covenant.org/), version 2.0, available at https://www.contributor-covenant.org/version/2/0/code_of_conduct.html.
+
+## Questions or Feedback?
+
+If you have any questions or feedback about the Code of Conduct, please contact [[email protected]](mailto:[email protected]).

CODE_OF_CONDUCT.md

@@ -0,0 +1,74 @@
+## Contributing to Flock
+
+Thank you for considering contributing to Flock! We welcome contributions from the community to help make this project even better.
+
+## How Can You Contribute?
+
+There are several ways you can contribute to Flock:
+
+1. **Code Contributions:**
+
+   - Implement new features or enhancements.
+   - Fix bugs and issues.
+   - Improve code quality through refactoring.
+   - Write and improve documentation.
+
+2. **Testing:**
+
+   - Test the project on different environments and report any issues.
+   - Write unit tests to ensure the stability of the codebase.
+
+3. **Feedback:**
+
+   - Provide feedback on the project's features, usability, and documentation.
+   - Suggest improvements or new ideas.
+
+## Getting Started
+
+If you're new to contributing to open-source projects, here's how you can get started:
+
+1. Fork the repository to your GitHub account.
+2. Clone the forked repository to your local machine:
+
+   ```bash
+   git clone https://github.com/yourusername/flock.git
+   cd flock
+   ```
+
+3. Create a new branch for your contribution:
+
+   ```bash
+   git checkout -b feature/new-feature
+   ```
+
+4. Make your changes, following the coding guidelines and best practices.
+5. Commit your changes:
+
+   ```bash
+   git commit -m "Add new feature"
+   ```
+
+6. Push your changes to your forked repository:
+
+   ```bash
+   git push origin feature/new-feature
+   ```
+
+7. Open a pull request (PR) against the `master` branch of the original Flock repository.
+
+## Coding Guidelines
+
+- Follow the [PEP 8](https://www.python.org/dev/peps/pep-0008/) style guide for Python code.
+- Write clear and concise commit messages.
+- Ensure your code passes any existing unit tests.
+- Include relevant documentation for your changes.
+
+## Code of Conduct
+
+Please note that by participating in this project, you agree to abide by our [Code of Conduct](CODE_OF_CONDUCT.md).
+
+## Questions or Feedback?
+
+If you have any questions or feedback, please don't hesitate to reach out by creating an issue or sending an email to [[email protected]](mailto:[email protected]).
+
+We appreciate your contributions and look forward to making Flock even better together!

CONTRIBUTING.md

@@ -0,0 +1,20 @@
+from pathlib import Path
+
+def create_symlink(target_path, symlink_path):
+    target = Path(target_path).resolve()
+    symlink = Path(symlink_path).resolve()
+    
+    if not symlink.exists():
+        symlink.symlink_to(target)
+        print(f"Symlink created: {symlink} -> {target}")
+    else:
+        print(f"Symlink already exists: {symlink}")
+
+# Replace these paths with your actual paths
+embedding_pipeline_src = "embedding_pipeline/src"
+prompt_engineering_src = "embedding_pipeline/src"
+main_flock_src = "flock/src"
+
+# Create symlinks
+create_symlink(embedding_pipeline_src, Path(main_flock_src, "embedding_pipeline"))
+create_symlink(prompt_engineering_src, Path(main_flock_src, "prompt_engineering"))

Untitled.ipynb

@@ -0,0 +1,3 @@
+# Embedding models and utilities
+# Storing embedding models and related utility functions
+# serialization and loading of trained models

flock/create_symlinks.py

@@ -0,0 +1 @@
+# Shared data preprocessing for all projects

flock/embedding_pipeline/src/main.py

@@ -0,0 +1 @@
+# embedding-related scripts

flock/embedding_pipeline/src/models/__init__.py

@@ -0,0 +1,2 @@
+# orchestrates the mebedding pipeline
+# loads data, preprocesses it, generates/embeds text

flock/prompt_engineering/src/main.py

@@ -0,0 +1,12 @@
+langchain
+tiktoken
+pypdf
+faiss-gpu
+chromadb
+transformers
+InstructorEmbedding
+sentence_transformers
+accelerate=0.21.0
+bitsandbytes
+xformers
+einops=0.6.1