Adding a command line parameter "-db-host" to optionally specify a re…

…mote machine to host the mysql database. This option is to be used with "--i-am-not-benchmarking". Changes were made in PerfOptions.php for the command line parsing, and DatabaseInstaller.php for installing the database. Drupal7, Mediawiki, and Wordpress are the targets which support this feature presently. For more info on how to run please see "Enabling_MySWL_Remote_Connection.txt". Resolving requested changes. Resolved requested changes. Added remote db support for all targets. Making $host local for cleanliness. updating oss-performance to work with recent changes to invariant sig… (facebookarchive#78) * updating oss-performance to work with recent changes to invariant signature * updating change to reflect comments * changing %s to %f for a numeric value Update siege blacklist to 3.0 Updated mediawiki to 1.28. Added setup script, php fpm support, modified batch run to be able to configure databases, fixed tc-print, and perf. Update fbcode option for use with buck. Add processor affinity setting "--cpu-fraction". Add no-jit option, and stat-cache option. Adding command line parameter to specify the number of HHVM Server Th… (facebookarchive#80) * Adding command line parameters to change the number of hhvm server threads as well as the benchmark concurrency. In order to use these options they must be paired with '--i-am-not-benchmarking' For HHVM Server Threads use: --hhvm-server-threads=<desiredNumber> The default is 100 For Benchmark Concurrency use: --benchmark-concurrency=<desiredNumber> The default is 200 * Adding command line parameter to specify the number of HHVM Server Threads as well as Benchmark Concurrency. These options are to be paired with '--i-am-not-benchmarking' For HHVM Server threads: --hhvm-server-threads=<number> Default is 100 For Benchmark Concurrency --benchmark-concurrency=<number> Default is 200 * Adding command line parameters to specify the number of ServerThreads (--server-threads=<NUM>) and the BenchmarkConcurrency (--benchmark-concurrency=<NUM>). Revising the method that these are set, using a setter method in PerfSettings.php for the BenchmarkConcurrency, set from PerfOptions.php. For ServerThreads we append the dhhvm.server.thread_count to the hhvmExtraArguments array. This is just a wrapper for the optional method of specifying via --hhvm-extra-arguments=-dhhvm.server.thread_count=<NUM>. * Made it so we do not have to use --i-am-not-benchmarking for --server-threads and --client-threads. * Resolved review comments. * Resolving requested changes. * Resolving requested changes, made pm.max_children variable.
kevand900 · Mar 22, 2017 · 324c79c · 324c79c
1 parent 9dd9706
commit 324c79c
Show file tree

Hide file tree

Showing 32 changed files with 522 additions and 200 deletions.
diff --git a/.gitignore b/.gitignore
@@ -5,3 +5,5 @@ vendor/
 core
 core.*
 composer.phar
+perf.data
+processed-perf.data
diff --git a/README.md b/README.md
@@ -26,12 +26,16 @@ As a regular user:
 
     composer.phar install # see https://getcomposer.org/download/
     hhvm perf.php --wordpress --php5=/path/to/bin/php-cgi # also works with php7
+    hhvm perf.php --wordpress --php=/path/to/bin/php-fpm # also works with php7
     hhvm perf.php --wordpress --hhvm=/path/to/hhvm
 
 Running with --hhvm gives some additional server-side statistics. It is usual
 for HHVM to report more requests than siege - some frameworks do call-back
 requests to the current webserver.
 
+:heavy_exclamation_mark: If you run with a debug build you may hit timeouts and
+other issues.
+
 Batch Usage
 ===========
 
@@ -45,18 +49,24 @@ See batch-run.json.example to get an idea of how to create batch-run.json.
 Requirements
 ============
 
+On Ubuntu you can run scripts/setup.sh.  This should provision your machine with
+everything you need to begin running the benchmarks.
+
+This installs:
+
 - composer
 - nginx
-- siege 2.x
+- siege (versions 2.x, or 3.1.x or 4.0.3)
 - unzip
 - A mysql server on 127.0.0.1:3306
 - hhvm
 
 I've been using the current versions available from yum on Centos 6.3. HHVM is
 required as this is written in Hack.
 
-Siege 3.x is not supported; as of writing, all 3.x releases have bugs that make
-it unable to correctly make the benchmark requests.
+Siege 3.0.x is not supported; as of writing, all 3.0.x releases have bugs that make
+it unable to correctly make the benchmark requests.  4.0.0, 4.0.1, 4.0.2 all
+automatically request resources on pages, and should not be used for benchmarking.
 
 The Targets
 ===========
@@ -188,14 +198,49 @@ loaded to provide a slightly more rounded workload.
 Profiling
 =========
 
+Perf
+----
 perf.php can keep the suite running indefinitely:
 
     hhvm perf.php --i-am-not-benchmarking --no-time-limit --wordpress --hhvm=$HHVM_BIN
 
 You can then attach 'perf' or another profiler to the running HHVM or php-cgi process, once the 'benchmarking' 
 phase has started.
 
-Direct support (especially for XHProf) is planned, but not yet implemented.
+There is also a script to run perf for you at the apropriate moment:
+
+    hhvm perf.php --i-am-not-benchmarking --wordpress --hhvm=$HHVM_BIN --exec-after-warmup="./scripts/perf.sh -e cycles"
+
+This will record 25 seconds of samples.  To see where most time is spent you can
+dive into the data using perf, or use the perf rollup script as follows:
+
+    sudo perf script -F comm,ip,sym | hhvm -vEval.EnableHipHopSyntax=true <HHVM SRC>/hphp/tools/perf-rollup.php
+
+In order to have all the symbols from the the translation cache you
+may need to change the owner of /tmp/hhvm-<PID>.map to root.
+
+
+TC-print
+--------
+TC-print will use data from perf to determine the hotest functions and
+translations.  Run the benchmark as follows:
+
+    hhvm perf.php --i-am-not-benchmarking --wordpress --hhvm=$HHVM_BIN --exec-after-warmup="./scripts/perf.sh -e cycles" --tcprint
+
+In order to have all the symbols from the the translation cache you
+may need to change the owner of /tmp/hhvm-<PID>.map to root.
+
+We process the perf data before passing it along to tc-print
+    sudo perf script -f -F hw:comm,event,ip,sym | <HHVM SRC>/hphp/tools/perf-script-raw.py > processed-perf.data
+
+tc-print is only built if the appropriate disassembly tools are available.  On
+x86 this is LibXed.  Consider building hhvm using:
+
+    cmake . -DLibXed_INCLUDE_DIR=<path to xed include> -DLibXed_LIBRARY=<path to libxed.a>
+
+Use tc-print with the generated perf.data:
+    <HHVM SRC>hphp/tools/tc-print/tc-print -c /tmp/<TMP DIR FOR BENCHMARK RUN>/conf.hdf -p processed-perf.data
+
 
 Contributing
 ============

diff --git a/base/BuildChecker.php b/base/BuildChecker.php
@@ -39,7 +39,7 @@ public static function Check(
       if ($skipKeys->contains($k)) {
         continue;
       }
-      invariant(is_array($v), $k.' is not an array');
+      invariant(is_array($v), '%s is not an array', $k);
       $v = self::MakeCheckedValue($v);
       if ($v['OK']) {
         continue;

diff --git a/base/DatabaseInstaller.php b/base/DatabaseInstaller.php
@@ -15,7 +15,9 @@ final class DatabaseInstaller {
   private ?string $username;
   private ?string $password = null;
 
-  public function __construct(private PerfOptions $options): void {}
+  public function __construct(private PerfOptions $options): void {
+    $this->configureMysqlAffinity();
+  }
 
   public function getUsername(): ?string {
     return $this->username ? $this->username : $this->databaseName;
@@ -35,9 +37,19 @@ public function setDumpFile(string $dump_file): this {
     return $this;
   }
 
+  public function configureMysqlAffinity(): void {
+    if ($this->options->cpuBind) {
+      exec("sudo taskset -acp ".$this->options->helperProcessors." `pgrep mysqld`");
+      print "You need to restart mysql after the benchmarks to remove the ";
+      print "processor affinity.\n";
+    }
+  }
+
   public function installDatabase(): bool {
     $db = $this->databaseName;
     $dump = $this->dumpFile;
+    $dbHost = $this->options->dbHost;
+
     invariant(
       $db !== null && $dump !== null,
       'database and dump must be specified',
@@ -47,7 +59,7 @@ public function installDatabase(): bool {
       return false;
     }
 
-    $conn = mysql_connect('127.0.0.1', $db, $db);
+    $conn = mysql_connect($dbHost, $db, $db);
     $db_selected = mysql_select_db($db, $conn);
     if ($conn === false || $db_selected === false) {
       $this->createMySQLDatabase();
@@ -76,7 +88,7 @@ public function installDatabase(): bool {
       '|'.
       $sed.
       Utils::EscapeCommand(
-        Vector {'mysql', '-h', '127.0.0.1', $db, '-u', $db, '-p'.$db},
+        Vector {'mysql', '-h', $dbHost.'', $db, '-u', $db, '-p'.$db},
       ),
       $output,
       $ret,
@@ -92,11 +104,17 @@ public function installDatabase(): bool {
   }
 
   private function getRootConnection(): resource {
-    print "MySQL admin user (default is 'root'): ";
-    $this->username = trim(fgets(STDIN)) ?: 'root';
-    fprintf(STDERR, '%s', 'MySQL admin password: ');
-    $this->password = trim(fgets(STDIN));
-    $conn = mysql_connect('127.0.0.1', $this->username, $this->password);
+    if ($this->options->dbUsername !== null
+        && $this->options->dbPassword !== null) {
+      $this->username = $this->options->dbUsername;
+      $this->password = $this->options->dbPassword;
+    } else {
+      print "MySQL admin user (default is 'root'): ";
+      $this->username = trim(fgets(STDIN)) ?: 'root';
+      fprintf(STDERR, '%s', 'MySQL admin password: ');
+      $this->password = trim(fgets(STDIN));
+    }
+    $conn = mysql_connect($this->options->dbHost, $this->username, $this->password);
     if ($conn === false) {
       throw new Exception('Failed to connect: '.mysql_error());
     }
@@ -105,7 +123,7 @@ private function getRootConnection(): resource {
 
   private function checkMySQLConnectionLimit(): void {
     $conn =
-      mysql_connect('127.0.0.1', $this->getUsername(), $this->getPassword());
+      mysql_connect($this->options->dbHost, $this->getUsername(), $this->getPassword());
     if ($conn === false) {
       throw new Exception('Failed to connect: '.mysql_error());
     }
@@ -135,7 +153,7 @@ private function createMySQLDatabase(): void {
       STDERR,
       '%s',
       "Can't connect to database ".
-      "(mysql -h 127.0.0.1 -p$db -u $db $db). This can be ".
+      "(mysql -h $this->options->dbHost -p$db -u $db $db). This can be ".
       "fixed for you.\n",
     );
     $conn = $this->getRootConnection();
@@ -162,7 +180,7 @@ private function createMySQLDatabase(): void {
       $conn,
     );
     mysql_query(
-      "GRANT ALL PRIVILEGES ON $edb.* TO '$edb'@127.0.0.1 ".
+      "GRANT ALL PRIVILEGES ON $edb.* TO '$edb'@'$this->options->dbHost' ".
       "IDENTIFIED BY '$edb'",
       $conn,
     );

diff --git a/base/HHVMDaemon.php b/base/HHVMDaemon.php
@@ -64,6 +64,9 @@ protected function getTarget(): PerfTarget {
 
   <<__Override>>
   protected function getArguments(): Vector<string> {
+    if ($this->options->cpuBind) {
+      $this->cpuRange = $this->options->daemonProcessors;
+    }
     $args = Vector {
       '-m',
       'server',
@@ -79,13 +82,21 @@ protected function getArguments(): Vector<string> {
       'Server.ErrorDocument404=index.php',
       '-v',
       'Server.SourceRoot='.$this->target->getSourceRoot(),
-      '-v',
-      'Eval.Jit=1',
+      '-d',
+      'hhvm.log.file='.$this->options->tempDir.'/hhvm_error.log',
       '-d',
       'pid='.escapeshellarg($this->getPidFilePath()),
       '-c',
       OSS_PERFORMANCE_ROOT.'/conf/php.ini',
     };
+    if ($this->options->jit) {
+      $args->addAll(Vector {'-v', 'Eval.Jit=1'});
+    } else {
+      $args->addAll(Vector {'-v', 'Eval.Jit=0'});
+    }
+    if ($this->options->statCache) {
+      $args->addAll(Vector {'-v', 'Server.StatCache=1'});
+    }
     if ($this->options->pcreCache) {
       $args->addAll(
         Vector {'-v', 'Eval.PCRECacheType='.$this->options->pcreCache},
@@ -107,6 +118,7 @@ protected function getArguments(): Vector<string> {
     if (count($this->options->hhvmExtraArguments) > 0) {
       $args->addAll($this->options->hhvmExtraArguments);
     }
+    $args->add('-dhhvm.server.thread_count='.$this->options->serverThreads);
     if ($this->options->precompile) {
       $bcRepo = $this->options->tempDir.'/hhvm.hhbc';
       $args->add('-v');
@@ -123,8 +135,6 @@ protected function getArguments(): Vector<string> {
       $args->add('Server.SourceRoot='.$sourceRoot);
     }
     if ($this->options->tcprint !== null) {
-      $args->add('-v');
-      $args->add('Eval.JitTransCounters=true');
       $args->add('-v');
       $args->add('Eval.DumpTC=true');
     }
@@ -273,46 +283,23 @@ public function stop(): void {
 
   public function writeStats(): void {
     $tcprint = $this->options->tcprint;
-    if ($tcprint) {
-      $conf = $this->options->tempDir.'/conf.hdf';
-      $args = Vector {};
-      $hdf = false;
-      foreach ($this->getArguments() as $arg) {
-        if ($hdf)
-          $args->add($arg);
-        $hdf = $arg === '-v';
-      }
-      $confData = implode("\n", $args);
-
-      file_put_contents($conf, $confData);
-      $args = Vector {$tcprint, '-c', $conf};
+    $conf = $this->options->tempDir.'/conf.hdf';
+    $args = Vector {};
+    $hdf = false;
+    foreach ($this->getArguments() as $arg) {
+      if ($hdf)
+        $args->add($arg);
+      $hdf = $arg === '-v';
+    }
+    $confData = implode("\n", $args);
 
+    file_put_contents($conf, $confData);
+    if ($tcprint) {
       $result = $this->adminRequest('/vm-dump-tc');
       invariant(
         $result === 'Done' && file_exists('/tmp/tc_dump_a'),
         'Failed to dump TC',
       );
-
-      if ($this->options->tcAlltrans) {
-        $alltrans = Utils::RunCommand($args);
-        file_put_contents('tc-all', $alltrans);
-      }
-
-      if ($this->options->tcToptrans) {
-        $new_args = new Vector($args);
-        $new_args->add('-t');
-        $new_args->add('100');
-        $toptrans = Utils::RunCommand($new_args);
-        file_put_contents('tc-top-trans', $toptrans);
-      }
-
-      if ($this->options->tcTopfuncs) {
-        $new_args = new Vector($args);
-        $new_args->add('-T');
-        $new_args->add('100');
-        $topfuncs = Utils::RunCommand($new_args);
-        file_put_contents('tc-top-funcs', $topfuncs);
-      }
     }
 
     if ($this->options->pcredump) {

diff --git a/base/NginxDaemon.php b/base/NginxDaemon.php
@@ -110,6 +110,9 @@ protected function getPidFilePath(): string {
 
   <<__Override>>
   protected function getArguments(): Vector<string> {
+    if ($this->options->cpuBind) {
+      $this->cpuRange = $this->options->helperProcessors;
+    }
     return Vector {
       '-c',
       $this->getGeneratedConfigFile(),