Getting Started: Overview

**If you haven't grab a local copy of our Examples, click here to learn how.

Code

You can create an Example.java anywhere in our Examples package.

Populate Example.java with the following lines.

import ai.preferred.venom.Crawler;
import ai.preferred.venom.Handler;
import ai.preferred.venom.Session;
import ai.preferred.venom.Worker;
import ai.preferred.venom.job.Scheduler;
import ai.preferred.venom.request.Request;
import ai.preferred.venom.request.VRequest;
import ai.preferred.venom.response.VResponse;

public class Example {

  private static class VenomHandler implements Handler {

    @Override
    public void handle(Request request, VResponse response, Scheduler scheduler, Session session, Worker worker) {
      String about = response.getJsoup().select(".sub-title b").text();
      System.out.println("ABOUT: " + about);
    }

  }

  public static void main(String[] args) throws Exception {
    try (Crawler c = Crawler.buildDefault().start()) {
      Request r = new VRequest("https://venom.preferred.ai");
      c.getScheduler().add(r, new VenomHandler());
    }
  }

}

Run the class with your favourite IDE.

You should now see the response:

ABOUT: Your preferred open source focused crawler for the deep web

Let's learn

How did the magic happen?

Let's break down what you have just done.

1. We created a Crawler and started it

try (Crawler c = Crawler.buildDefault().start()) {
  ...
}

2. We created a Handler to query the element from a response

String about = response.getJsoup().select(".sub-title b").text();
System.out.println("ABOUT: " + about);

3. We submitted the request and attached a handler

Request r = new VRequest("https://venom.preferred.ai");
c.getScheduler().add(r, new VenomHandler());

The request has been scheduled, and the response will be handled by VenomHandler

Congratulations

Now you have created a your very first basic Venom crawler.