-
Notifications
You must be signed in to change notification settings - Fork 0
/
README.new
234 lines (146 loc) · 6.43 KB
/
README.new
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
Finals Club Tools & Script repo
================================
I. Introduction
--------------------------------------
The Finals Club Foundation has been providing free and
open education resources to students since 2008. As of
2011, FCF has made all services and tools free and open
to those who would like to re-deploy services offered by
FCF. This document shows how all services are deployed
in production and what is used for best practice in a
real world environment.
#### Current Structure ####
+ /scripts is where all scripts and utilities are kept. All
scripts will be re-written in ruby for more abstractness.
+ /node-proxy is the old implementation of node-http-proxy
that was used for routing traffic around the server.
+ /nginx_conf contains our nginx configuration files.
II. Basic Model
--------------------------------------
All services deployed by the Finals Club Foundation are
based upon the following stack:
________________________________________________
| | | |
| App.(n) | App.(n) | etc.. |
| | | |
------------------------------------------------
| | |
| Node.Js | django / python |
| | |
------------------------------------------------
| |
| GNU/Linux (ubuntu / Debian ) |
| |
|________________________________________________|
Base: GNU/Linux:
--------------------------------------
All servers that run services for the Finals Club Foundation
start off with a generic GNU/Linux stack with:
* Ubuntu-Server 11.10+
* Mongodb-server
* Node.Js V.0.6.13+
* npm current
* Monit (optional, but recommended for basic monitoring).
* Dev. Tools: gcc, git, curl, make
* Node Modules for each app. / service
* Nginx 1.18 Web Server
* Django-1.4
* PostgreSQL-1.9+
While the previous is an extensive list, it is not everything. See
what is included with ubuntu-server 11.10+ for other tools are
on our servers.
Middle Node.js:
--------------------------------------
Node.js is a new system for writing and deploying web applications
that are able to scale and update user information in real time. We
used Node.js because it is a new technology that is in its infancy, but
is becoming commonly used in the development of new web applications and tools.
Etherpad-lite (Collaborative editor used in Finalsclub.org) is one of the many
components in our many services that are written in Node.Js.
Middle Django / Python:
--------------------------------------
Karma Notes http://karmanotes.org is written using the django web framework
and uses uWSGI + Nginx to provide resources. Nginx was used instead of Apache
because of Apache's age and security shortcomings.
Top: App.(n)...
--------------------------------------
Some of our apps / services are written for node.js
* Finals Club.org
* Node HTTP proxy (for routing traffic around the server).
III. Deployment:
--------------------------------------
a. Finals Club
----------------------
Deployment is fairly straight forward.
1. install git, curl, npm, node.js, mongodb-server, gcc, g++
2. git clone https://github.com/FinalsClub/FinalsClub.git
Please put this in a common Dir. In production we use:
/var/www/$(Service)
If the server is shared with users you do not directly know, then
you should create a "system user" and home for your web services.
Creating a special user and home for services allows server admins.
to track the activity of services and can prevent damage to the system
if the App. / service is compromised.
3. git checkout devel # In FinalsClub src. Root.
4. git submodule init && git submodule update
5. Move "setup.sh" to another location and update to
reflect the configuration of your environment.
6. if everything is deployed and setup properly (including sys. user
account), you should be able to run "sudo start&" as the user that
was created.
b. Karma Notes
----------------------
1. Pull the latest code from : https://github.com/FinalsClub/djKarma.git
djkarma is located in : /var/www/djKarma
djkarma User: djkarma
2. install requirements:
pip install -r $(djKarma REPO)/requirements.txt
b1.) Postgresql
1. Install dependencies.
sudo apt-get install postgresql python-psycopg2 python-pip
2. Reset postgres password
sudo passwd postgres
3. “Su” as postgres
sudo su postgres
4. Create DB User “djkarma”
sudo -u postgres createuser -P djkarma
5. Create DB from template
psql template1
6. Add line to config file
echo “ local karmanotes djkarma md5” >> /etc/postgresql/9.1/main/pg_hba.con
7. Restart postgresql server
sudo service postgresql restart
At this point, we have several options for dealing with databases. If we have a backup to restore from, we can:
As user postgres:
psql < kn_backup.sql # Note this only works from a full dump of a server.
$(djKarma REPO)/manage.py syncdb
b2.) Apache Solr
See http://lucene.apache.org/solr/api-3_6_0/doc-files/tutorial.html
All solar deployments use an internal tarball of solr (includes config).
Apache solr is deployed to : /var/www/solr
init. run : ./manage.py rebuild_index # Answer yes
To RUN: /var/www/scripts/solr.sh
The solr.sh script can be found in the fc_tools repo. Also note that this script
must ran under the solr user that you create (ours is solar).
NOTE: All web services are located in /var/www/*.
/var/www/ is owned by the web group and each service MUST BE OWNED AND RAN AS ITS
OWN USER!!! Failure to do so will compromise security. Please see index for more
info.
b3.)Celery
Make sure the following are present or installed on the server:
* Python-pip
* fc_tools git repo https://github.com/FinalsClub/fc_tools
To actually install celery, we do:
sudo pip install django-celery
To ensure it runs on every server restart, place the following in their respective
places:
* $(fc_tools REPO)/scripts/celeryconfig -> /etc/default/celeryconfig
* $(fc_tools REPO)/scripts/celeryd -> /etc/init.d/celeryd
To restart celery, run the following as user celery:
/etc/init.d/celeryd start
IV. Backup / Management
--------------------------------------
To backup our static files and DB(s), we use fairly generic scripts. In the future
we are looking to bup for clean incremental backups that will not take up so much space.
To use our backup.sh script, you need to define all missing env. vars and use the following structure:
At present, we use kn-backup.sh and fc-backup.sh