Skip to content

GCE Feeder Box Tutorial

Jump to bottom
Admin9705 edited this page May 12, 2018 · 94 revisions

CLICKING the STAR in the UPPER RIGHT promotes our PROJECT on GITHUB!

NOTE: YOU MUST BE USING PGDEV 5.080+ for now

Table of Contents: GCEFB

About GCEFB

  • CleanUp Author: Admin9705

Special thanks to John Doe for his work on experimenting with new methods and beta testing supertransfer2. GITHUB: xxx

Compute Engine

Basically fill out here what it is

Establishing Billing & Basic Parameters

  1. Ensure to start off by selecting the right category.

  1. When the page first loads, start off by selecting the Compute Engine.

  1. If you have not enabled your billing, you will be prompted to do so. Be careful with your usage and understand that you can be liable for any money owed.

  1. Create a Compute Engine instance:

  1. Recommend Specifications are as Followed!
  • Zone: (pick one closest to your usenet server):
    • us-central1-a (Iowa)
    • europe-west1-b (Belgium)
    • asia-east1-a (Taiwan)
    • we are picking these locations because they have 2.6ghz cores instead of 2.2ghz
  • 8-12 Cores
    • If you go above 8 cores, you will have to upgrade your account
    • This means you will have to remember to cancel your server when credits run out
  • 7-11G Memory
  • Ubuntu 16.04 LTS
    • Boot disk: 10G
    • If google added NVMe drivers to 18.04 by the time you are reading this, choose 18.04 instead.
  1. Select Local SSD Scratch Disk. To start, the option is somewhat hidden; but luckily we have a picture here for you where to start.

  1. From here, you going to create x2 Local SSD Scratch Disks. Make sure that you CREATE 2 and make sure they are NVMe versions!

Optional Security

Enter in your public ssh key so you can access the box through your own ssh client.

When you are finished with your establishing your Compute Engine, visit the front page and you should see the something of the following!

Installing PlexGuide GCE Feeder Edition

  1. First install PlexGuide by visiting here: Install PlexGuide

  2. Go through the Install Process and Select the GCE Feeder Edition. If no errors, the program will deploy most of your programs by default!

WARNING - BONEHEAD ALERT - WARNING:

If you get this error below afterwards, it means there is a 99.9 percent chance that you FAILED to create the 2 additional drives that are required.

Configuring Your Static IP & Firewall

First, you must configure your network. Please view the picture below on what to click.

Next, you have to setup your external IP address. If you fail to do this, you will not be able to access the programs!

From there, you will click a top to reserve a static ip address.

Next, you will click firewall rules and add a new one.

Ensure to setup you information as shown below. In regards to the ports for better security, you can copy this line: tcp:9000; tcp:7878; tcp:8989; tcp:6789; tcp:8080; tcp:8000; tcp:7999

If you are having problems, just select the all option; but just keep in mind that this results in poor security.

Now test by going to your programs. IPv4:9000 should work by default due to it being Portainer. If you just applied the rules, wait up-to 1 minute for the modified Google Firewall rules to kick in.

Configuring Your Compute Engine

Make a Teamdrive at drive.google.com and name it Media. [Instructions]

Note: If you cannot create a teamdrive, do not see it, or have errors; [Follow This]

In it, create your directories: movies and tv Configure rclone with the teamdrive: rclone config

  1. n/s/q> n

  2. name> teamdrive

  3. storage> type --> drive

  1. client_id> [PRESS ENTER]

  2. client_secret> [PRESS ENTER]

  3. scope> 1

  4. root_folder_id> [PRESS ENTER]

  5. service_account_file> [PRESS ENTER]

  6. headless browser question y/n > n

  7. enter verification code> [PASTE FROM BROWSWER]

  8. configure as a Team Drive y/n > y

  9. enter a Team Drive ID> 1

Verify that your teamdrive is configured properly with rclone touch teamdrive:/test123 && echo "rclone OK" You should see a file called test123 in your Media teamdrive at drive.google.com/

Configure rclone cache with your teamdrive with rclone config

name> rcache
Storage> cache
remote> teamdrive:
[ skip plex related questions ]
chunk_size> 10M
info_age> 1h
chunk_total_size> 500M

Install rclone cache & unionfs service daemons: ansible-playbook /opt/plexguide/ansible/plexguide.yml --tags rcache

chown -R 1000:1000 /root
chmod 775 /root/.config/rclone/rclone.conf

Verify rclone cache is mounted with systemctl status rcache. If it isn't, check the error message and restart it with systemctl restart rcache.

Verify that there are teamdrive directories you made eariler in /mnt/rcache

Verify unionfs is mounted with systemctl status unionfsc. If it isn't, check the error message and restart it with systemctl restart unionfsc.

Installing SuperTransfer2

Now load up plexguide and go to PG Settings & Tools > SuperSpeeds > Multi-SA SuperTransfer2 req: teamdrives. Now press any key and y to start configuraton:

Load up http://yourip:8000 ready to upload your SA.json files which you will be creating next.

How to create your SA .json files:

Go to https://console.developers.google.com/ and make sure you have project select that has Google Drive API enabled on it. Now follow the steps below:

  1. Go to “IAM & admin” -> “Service Accounts”.

  2. Use the “Create Service Account” button. Fill in “Service account name” with SA1 for example. Tick “Furnish a new private key” - select “Key type JSON”. Leave everything else as default.

  3. Save your .json file and do this another 39 times. Call your next on SA2 for example and so on.

Hint: if you have many keys, it's easier to compress them into an archive, upload that, then extract server-side

Hint: If port 8000 is closed or you wish to upload keys securely, Transfer json keys directly into: /opt/appdata/plexguide/supertransfer

Visit: https://github.com/Admin9705/PlexGuide.com-The-Awesome-Plex-Server/wiki/SuperTransfer2

Troubleshooting

Troubleshooting:

  1. Make sure user 1000 can read and write into /root/.config/rclone/rclone.conf
  2. Make sure there is a proper config in /root/.config/rclone/rclone.conf (sometimes, the config ends up in ~/.config/rclone/rclone.conf)
  3. Make sure the line user_allow_other is uncommented in /etc/fuse.conf

PG Forum Support: GCEFB

https://plexguide.com/forums/pg-3rd-party-programs.148/

Visit our Website By Click the Picture Below! N|Demo

N|Solid N|Solid

Supertransfer2

Pre-requisite

**Google Drive Service Accounts ** Go to http://console.developers.google.com and navigate to the credentials area to create new credentials - the quickest way is through the 'Manage Service Accounts' page.

Service Account Name: name account (e.g. GSA1, GSA2 etc) Role: leave blank Check 'Furnish New Private Key' Check 'JSON' under key type Check 'Enable G Suite Domain-wide Delegation'

And save - this will download the JSON to your HDD. Repeat this 10-50 times (that's plenty really)

Configure SuperTransfer2

  • Optional - Install BBR Kernel Mod from the Plexguide menu under Settings and Tools and select Kernel Mods then its Option 1.

---- OLD NOTES

SSH into your box and switch to root with sudo su

Update your server with apt-get update && apt-get upgrade -y

Mount your first NVMe drive to /mnt, and the other to /nvme2. We will be using /mnt as the directory to download files to, and /nvme2 as the directory that files are unpacked to and uploaded to gdrive. (note: we will be symlinking directories in /nvme2, not writing to it directly)

Make sure you have both NVMe disks on your server with lsblk | grep nvme. You should see two unmounted NVMEe disks named nvme0n1 and nvme0n2.

Mount your NVMe disks.

mkfs.ext4 -F /dev/nvme0n1
mount -o discard,defaults,nobarrier /dev/nvme0n1 /mnt
chmod a+w /mnt
echo UUID=`blkid | grep nvme0n1 | cut -f2 -d'"'` /mnt ext4 discard,defaults,nobarrier,nofail 0 2 | tee -a /etc/fstab

mkdir -p /nvme2
mkfs.ext4 -F /dev/nvme0n2
mount -o discard,defaults,nobarrier /dev/nvme0n2 /nvme2
chmod a+w /nvme2
echo UUID=`blkid | grep nvme0n2 | cut -f2 -d'"'` /nvme2 ext4 discard,defaults,nobarrier,nofail 0 2 | tee -a /etc/fstab

Verify if they mounted properly.

ls /mnt | grep -q 'lost+found' && echo "nvme1 mount OK!" || echo "nvme1 mount FAIL"
ls /nvme2 | grep -q 'lost+found' && echo "nvme2 mount OK!" || echo "nvme2 mount FAIL"

Install Plexguide with wget -qO- plexguide.com/install.sh | bash

- Settings:
  - No second HDD

- Install:
  - Sonarr
  - Radarr
  - Sabnzbd

Verify the ports are open: for port in 7878 8989 8080 9000; do nc -w1 -zv $(curl -s icanhazip.com) $port; done

Create passwords for each app and restart the containers: for container in portainer radarr sonarr sabnzbd; do docker restart $container; done

Make a symlink for /mnt/move and /mnt/sab/complete to /nvme/... This will allow usenet articles to unpack and upload a little faster, since it's on a separate disk. You can repeat this process with nzbget, if you prefer it over sabnzbd.

mv /mnt/move /nvme2/move
ln -s /nvme2/move /mnt

mkdir -p /nvme2/sab/complete
rm -r /mnt/sab/complete
ln -s /nvme2/sab/complete /mnt/sab/

chown -R 1000:1000 /mnt
chown -R 1000:1000 /nvme2

NOTE: no menu items for these yet. Please install manually.

Install rclone beta: curl -s https://rclone.org/install.sh | bash -s beta

Home - https://plexguide.com

Installing PlexGuide

  1. PlexGuide Install Information

Preplanning & Information

  1. PG Folder Structure

Domain and Port Control

  1. CloudFlare Tunnel

Primary Applications

  1. Plex

Useful Links

  1. PG YouTube Channel
Clone this wiki locally