Updated hbc to work with NTL 10.5.0 - closes #3

qdm12 · qdm12 · commit 83c717bc9097 · 2017-09-04T21:54:23.000-04:00
Checked Vagrant works with the latest NTL 10.5.0 and HElib. Readme.md has been updated and Cygwin has been checked as non-working still (see readme.md). The level of the multiplier circuit has been increased by 1 to prevent a warning message from displaying (although it was working). In the end, everything works with Vagrant and should work by manually installing everything as well.
diff --git a/README.md b/README.md
@@ -1,12 +1,14 @@
-# Homomorphic binary circuits - _hbc_
+# Homomorphic binary circuits - *hbc*
 
 *This project was developed as my Master final year project at Imperial College London.*
 
-*This project is still maintained by Quentin McGaw (email: quentin . mcgaw at gmail . com)*
+*This project is still maintained (Sept 2017) by Quentin McGaw (email: quentin . mcgaw at gmail . com)*
+
 
 ## 1. What is it, in *one* line?
 It is an **API** of homomorphic binary operations such as binary comparison or binary Euclidian division using the library _HElib_.
 
+
 ## 2. What is in there?
 - It is written in C++ and is cross-platform with Vagrant
 - The **core API** is in *src/he.cpp*.
@@ -18,29 +20,43 @@ It is an **API** of homomorphic binary operations such as binary comparison or b
 - There is this complete, detailed and updated **README.md** file.
 - There are links in this readme to my **presentation** (VERY USEFUL) and my detailed report (WAY TOO LONG).
 
+
 ## 3. What does it run ?
 - It runs the code in *main.cpp* which execute the unit tests on all the homomorphic circuits implemented.
 - You can change the main.cpp with your code by inspiring from the tests.
 - You can also add circuits in *he.cpp* and then add corresponding tests, and **tell me** you want to contribute !
 
+
 ## 4. What does it require ?
 - Practically:
     - A Linux/Windows/OSX computer
     - At least 3GB of RAM and 2 CPU cores
     - An internet connection
-    - CPU with Hardware virtualization tech (you probably have it don't worry)
-- In terms of software (although this is automatically installed): ![hbc dependencies diagram][dependencies_jpg]
+    - CPU with Hardware virtualization tech ideally (you probably have it don't worry)
+- In terms of software (although this is automatically installed), here are the dependencies:
+
+
+   | Program or Library | Requirement 1 | Requirement 2 | Requirement 3 | Requirement 4 | Requirement 5 |
+   | ------------------ | ------------- | ------------- | ------------- | ------------- | ------------- |
+   | hbc                | g++           | make          | HElib         | c++11         |
+   | HElib              | g++           | make          | git           | libboost      | NTL 10.5.0    |
+   | NTL 10.5.0         | g++           | make          | GMP 6.1.2     |
+   | GMP 6.1.2          | g++           | make          | m4            | perl          |
+
 
 ## 5. Documentation
 - This readme file
 - **Presentation** available on [Dropbox][dropbox_pres], on [Github][github_pres] and on [Youtube](https://www.youtube.com/watch?v=n-adgQWZYxI) where I present it.
 - Report available on [Github][github_report] or on [Dropbox][dropbox_report]
 - Comments in the source code, especially in _he.cpp_
 
+
 ## 6. Abstract ##
 This project concerns the research and development of a real-use application of homomorphic encryption for cloud computing. The application takes advantage of the various possibilities and limitations of present homomorphic encryption schemes and programming libraries to remain usable in terms of time. The foundations of the application rely on the design of binary operations using homomorphic encryption. All the binary logic gates and various binary blocks were developed and adapted to provide enough functionalities to the application. The project focuses on providing features to cloud computing such as calculating averages on large amounts of encrypted numbers in a relatively short and decent time. The result is an application program interface written in C++ allowing to perform various operations on integers. It thus shows homomorphic encryption can be used today for simple operations if the security is more important than the speed of execution. 
 
+
 ## 7. How do I run it?
+
 ### 7.1 Using Vagrant (easiest, compatible with all, most flexible)
 1. Install git on your computer
     - `apt-get install -y git` for Linux machines
@@ -88,87 +104,120 @@ This basically launches an Ubuntu-based virtual machine with only what is necess
             8. Install it with `make install`
             9. *Optionally*, check it with `make check`
             10. Go back and remove used files with `cd .. && rm -fr gmp-6.1.2*`
-            10. Download NTL with `curl http://www.shoup.net/ntl/ntl-9.9.1.tar.gz > ntl-9.9.1.tar.gz`
-            11. Extract it and go to its directory with `tar -xvzf ntl-9.9.1.tar.gz && cd ntl-9.9.1/src`
+            10. Download NTL with `curl http://www.shoup.net/ntl/ntl-10.5.0.tar.gz > ntl-10.5.0.tar.gz`
+            11. Extract it and go to its directory with `tar -xvzf ntl-10.5.0.tar.gz && cd ntl-10.5.0/src`
             12. Configure it with `./configure NTL_GMP_LIP=on`
             13. Build it with `make`
             14. Install it with `make install`
-            15. Go back and remove used files with `cd ../.. && rm -fr ntl-9.9.1*`
+            15. Go back and remove used files with `cd ../.. && rm -fr ntl-10.5.0*`
             16. Clone HElib with with `git clone https://github.com/shaih/HElib.git`
             17. Go to its src directory `cd HElib/src`
             18. Build it with `make`
             19. *Optionally*, check it with `make check` and test it with `make test`.
             20. Go back to the working directory with `cd ../..`
         - Other Linux OSes
-            1. Install the libaries with (add `*sudo*` maybe) `apt-get install git g++ m4 perl libboost-all-dev`
+            1. Install the libaries with (add `sudo` maybe) `apt-get install git g++ m4 perl libboost-all-dev`
             2. Download GMP with `wget https://gmplib.org/download/gmp/gmp-6.1.2.tar.bz2`
             3. Extract it and go to its directory with `tar -xvjf gmp-6.1.2.tar.bz2 && cd gmp-6.1.2`
             4. Configure it with `./configure`
             5. Build it with `make`
             6. Install it with `make install`
             7. *Optionally*, check it with `make check`
             8. Go back and remove used files with `cd .. && rm -fr gmp-6.1.2*`
-            9. Download NTL with `wget http://www.shoup.net/ntl/ntl-9.9.1.tar.gz`
-            10. Extract it and go to its directory with `tar -xvzf ntl-9.9.1.tar.gz && cd ntl-9.9.1/src`
+            9. Download NTL with `wget http://www.shoup.net/ntl/ntl-10.5.0.tar.gz`
+            10. Extract it and go to its directory with `tar -xvzf ntl-10.5.0.tar.gz && cd ntl-10.5.0/src`
             11. Configure it with `./configure NTL_GMP_LIP=on`
             12. Build it with `make`
             13. Install it with `make install`
-            14. Go back and remove used files with `cd ../.. && rm -fr ntl-9.9.1*`
+            14. Go back and remove used files with `cd ../.. && rm -fr ntl-10.5.0*`
             15. Clone HElib with with `git clone https://github.com/shaih/HElib.git`
             16. Go to its src directory `cd HElib/src`
             17. Build it with `make`
             18. *Optionally*, check it with `make check` and test it with `make test`.
             19. Go back to the working directory with `cd ../..`
         - Cygwin 32 bit and 64 bit
-            1. It will not work for mysterious reasons, sorry ! Use Vagrant !
+			- It will fail when you try to install NTL with the NTL_CMP_LIP=on because cygwin does not find the -lgmp libary for some reason.
+			- So just switch to use Vagrant. I might be missing something but there is no point digging to deep here I believe.
+			- Here would be the procedure:
+               1. Close any Cygwin processes running
+               2. Download the right Cygwin installer:
+                  - [Cygwin 32 bit](http://www.cygwin.com/setup-x86.exe)
+               	  - [Cygwin 64 bit](http://www.cygwin.com/setup-x86_64.exe)
+               3. Run the Cygwin installer previously downloaded
+               4. Click *Next >*, *Next >*, *Next >*, *Next >*, *Next >*, *Next >*
+               5. With the help of the search bar, select the following packages (only the Devel is necessary):
+                  - git: Distributed version control system
+                  - gcc-g++: GNU Compiler Collection (C++)
+                  - make: the GNU version of the 'make' utility
+                  - m4: GNU implementation of the traditional Unix macro processor
+                  - perl: Perl Programming language interpreter
+                  - libboost-devel: Boost C++ libraries
+                  - gmp: Library for arbitrary precision arithmetic
+               6. Click on *Next >*, *Next >* and wait for the installation to finish and then close the window.
+               7. Lauch a Cygwin terminal
+               8. Download NTL with `wget http://www.shoup.net/ntl/ntl-10.5.0.tar.gz`
+               9. Extract it and go to its directory with `tar -xvzf ntl-10.5.0.tar.gz && cd ntl-10.5.0/src`
+               10. Configure it with `./configure NTL_GMP_LIP=on`			
+               11. Build it with `make`
+               12. Install it with `make install`
+               13. Go back and remove used files with `cd ../.. && rm -fr ntl-10.5.0*`
+               14. Clone HElib with with `git clone https://github.com/shaih/HElib.git`
+               15. Go to its src directory `cd HElib/src`
+               16. Build it with `make`
+               17. *Optionally*, check it with `make check` and test it with `make test`.
+               18. Go back to the working directory with `cd ../..`
 4. Build hbc
     - With the Makefile provided (compatible will **all** platforms).
         1. Build it with `make hbc`
-    - Manually (Works on everything, even Cygwin)
+    - Manually
         1. Create the directory objects `mkdir -p objects`
         2. Compile the API `g++ -c src/he.cpp -I HElib/src -o objects/he.o`
         3. Compile the helper functions `g++ -c src/helper_functions.cpp -o objects/helper_functions.o`
         4. Compile the various tests
-            * `g++ -c src/TEST_GATES.cpp -I HElib/src -o objects/test_gates.o`
-            * `g++ -c src/TEST_CIRC_COMB.cpp -I HElib/src -o objects/test_circ_comb.o`
-            * `g++ -c src/TEST_CIRC_SEQ.cpp -I HElib/src -o objects/test_circ_seq.o`
-            * `g++ -c src/TEST_CIRC_ARITHM.cpp -I HElib/src -o objects/test_circ_arithm.o`
+            - `g++ -c src/TEST_GATES.cpp -I HElib/src -o objects/test_gates.o`
+            - `g++ -c src/TEST_CIRC_COMB.cpp -I HElib/src -o objects/test_circ_comb.o`
+            - `g++ -c src/TEST_CIRC_SEQ.cpp -I HElib/src -o objects/test_circ_seq.o`
+            - `g++ -c src/TEST_CIRC_ARITHM.cpp -I HElib/src -o objects/test_circ_arithm.o`
         5. Compile the main.cpp file `g++ -c src/main.cpp -I HElib/src -o objects/main.o`
-        6. Compile the objects into hbc `g++ objects/*.o HElib/src/fhe.a -o hbc -L/usr/local/lib -lntl -lgmp -lm`
+        6. Compile the objects into *hbc* `g++ objects/*.o HElib/src/fhe.a -o hbc -L/usr/local/lib -lntl -lgmp -lm`
 5. Run hbc
-    - If on Vagrant, `cd /vagrant` first
-    - Run it with `./hbc` && Enjoy ! (Careful about having enough **RAM**)
+    - Run it with `./hbc` (Careful about having enough **RAM**)
     - You can also build it and run the new build with `make hbcNrun`
-6. RAM considerations **IMPORTANT**
-    - To run the default hbc program, you need at least 3GB of ram.
-    - This is because the average operation (arithmetic tests) uses about 2GB of ram for 2 bits.
+
+	
+## 8. RAM considerations **IMPORTANT**
+    - To run the default hbc program, you need at least 3GB of RAM.
+    - This is because the average operation (arithmetic tests) uses about 2GB of RAM for 2 bits.
     - Note that you can comment it out in the main.cpp or TEST_ARITHMETIC.cpp 
       and stick to tests of simpler circuits such as the multiplication which 
-      only require about 0.7 - 1GB of ram. You can modify the amount of RAM of 
-      Vagrant in the vb.memory field, which is set to 2600MB by default.
-    - To monitor the RAM on Vagrant, open a new host terminal, go to the working
-      directory and use `vagrant ssh -c htop`.
-7. CPU considerations
+      only require about 0.7 - 1GB of RAM. 
+	- For **Vagrant**, you can modify the amount of RAM in the **vb.memory** field, 
+	  which is set to **2600MB** by default. To monitor the RAM usage, open a new 
+	  host terminal, go to the working directory and use `vagrant ssh -c htop`.
+
+	  
+## 9. CPU considerations for Vagrant
     - By default, the Vagrant VM uses 2 cores of your CPU (vb.cpus = 2) so that
       you can run hbc and also monitor the RAM with another `vagrant ssh`.
     - You can also run more instances of hbc if you have more than two cores available.
       With Vagrant, just set vb.cpus to 3 for example, log in with `vagrant ssh` on different
-      host terminals and run hbc (provided you have enough ram to run both obviously).
+      host terminals and run hbc (provided you have enough RAM to run both obviously).
 
+	  
+## 10. Remove and uninstall ##
 
-## 8. Remove and uninstall ##
-### 8.1 With Vagrant
-Just enter `vagrant destroy` from your host machine in the working directory
+### 10.1 With Vagrant
+Just enter `vagrant destroy` from your host machine in the working directory.
 
-### 8.2 With Debian and Ubuntu
+### 10.2 Otherwise
 Use the makefile and run `make deepclean` which uninstalls and delete:
 - hbc
 - HElib, NTL, GMP
 - perl, m4, git, gcc-g++ and libboost-all-dev and purge them.
 Only the makefile will remain in the folder.
 
 
-## 9. Acknowledgements ##
+## 11. Acknowledgements ##
 Credits to **Shai Halevi** for HElib obviously and thanks for the quick help tips in the Issues section!
 
 Thanks to **Dr. Wei Dai** (Imperial College London) for introducing me to homomorphic encryption
@@ -182,15 +231,15 @@ Thanks to **Wei Dai** from the Vernam Group at Worcester Polytechnic Institute f
 Thanks to **my dad**, **mother** and **step-mother** for their continuous support throughout my studies.
 
 
-## 10. Contribution
+## 12. Contribution
 You're welcome to contribute to this repository if you find any better circuits or other circuits and implement them !
 
 Just send me an email (see my address at the start) and I will add you as a contributor.
 
 Don't hesitate to contact me if you have any question as well.
 
 
-## 11. Future Work (ideas crossing my mind) ####
+## 13. Future Work (ideas crossing my mind) ####
 - Use *= instead of multiplyBy when the level is very low as it is faster. multiplyBy uses relinearization which serves to reduce the size of ciphertexts. This is great for complex circuits but takes a longer time than *= for simple circuits.
 - Add circuits from [here](http://www.aoki.ecei.tohoku.ac.jp/arith/mg/algorithm.html) to the core API *he.cpp*.
 - Other ideas are in chapter 9: _Future work_ of my report.
diff --git a/Vagrantfile b/Vagrantfile
@@ -23,20 +23,21 @@ Vagrant.configure(2) do |config|
     cd gmp-$GMP_V
     ./configure
     make
-    make install
+    sudo make install
     make check
+	cd ..
     rm -fr gmp-$GMP_V*
     unset GMP_V
     
     #Installing NTL
-    NTL_V=9.9.1
+    NTL_V=10.5.0
     wget http://www.shoup.net/ntl/ntl-$NTL_V.tar.gz
     tar -xvzf ntl-$NTL_V.tar.gz
     cd ntl-$NTL_V/src
     ./configure NTL_GMP_LIP=on
     make
-    make install
-    cd ..
+    sudo make install
+    cd ../..
     rm -fr ntl-$NTL_V*
     unset NTL_V
     
@@ -49,8 +50,8 @@ Vagrant.configure(2) do |config|
         cd HElib/src
         make
         #make check
+		#1 of the check of HElib fails for some reason
         #make test
-        #Test of HElib fails for some reason
         cd ../..
     fi
    
diff --git a/docs/dependencies.jpg b/docs/dependencies.jpg
diff --git a/makefile b/makefile
@@ -1,16 +1,16 @@
 GMP_V = 6.1.2
-NTL_V = 9.9.1
+NTL_V = 10.5.0
 COMPILER = g++
 
-hbc : mkdir objects/he.o objects/helper_functions.o \
+hbc : createdirs objects/he.o objects/helper_functions.o \
 		objects/test_gates.o objects/test_circ_comb.o objects/test_circ_seq.o \
 		objects/test_circ_arithm.o \
 		objects/main.o
 	$(info )
 	$(info Compiling hbc...)
 	$(COMPILER) -std=c++11 objects/*.o HElib/src/fhe.a -o hbc -L/usr/local/lib -lntl -lgmp -lm
 
-mkdir :
+createdirs :
 	mkdir -p objects
 	
 objects/he.o : src/he.cpp src/he.h
@@ -50,7 +50,7 @@ gmp : ini
 	#cd gmp-$(GMP_V) && ./configure ABI=64
 	cd gmp-$(GMP_V) && ./configure
 	cd gmp-$(GMP_V) && make
-	cd gmp-$(GMP_V) && make install
+	cd gmp-$(GMP_V) && sudo make install
 	cd gmp-$(GMP_V) && make check
 	rm -fr gmp-$(GMP_V)*	
 
@@ -61,7 +61,7 @@ ntl : ini gmp
 	#cd ntl-$(NTL_V)/src && ./configure NTL_GMP_LIP=on CFLAGS="-O2 -m64"
 	cd ntl-$(NTL_V)/src && ./configure NTL_GMP_LIP=on
 	cd ntl-$(NTL_V)/src && make
-	cd ntl-$(NTL_V)/src && make install
+	cd ntl-$(NTL_V)/src && sudo make install
 	rm -fr ntl-$(NTL_V)*
 	
 HElib : gmp ntl
diff --git a/src/TEST_CIRC_ARITHM.cpp b/src/TEST_CIRC_ARITHM.cpp
@@ -57,7 +57,7 @@ bool TEST_CIRC_ARITHM::test_MULTIPLY(){
 
 		switch(bits){
 			case 2:
-				params.L = 8; //for 1024 slots
+				params.L = 9; //for 1024 slots
 				break;
 			case 3:
 				params.L = 13; //for 1024 slots
