Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is there a method to return ALL the ids in the index? #93

Open
nelsonic opened this issue Dec 7, 2015 · 7 comments
Open

Is there a method to return ALL the ids in the index? #93

nelsonic opened this issue Dec 7, 2015 · 7 comments
Assignees
Labels

Comments

@nelsonic
Copy link
Member

nelsonic commented Dec 7, 2015

http://stackoverflow.com/questions/17497075/efficient-way-to-retrieve-all-ids-in-elasticsearch

@nelsonic nelsonic self-assigned this Dec 7, 2015
@nelsonic nelsonic mentioned this issue Dec 7, 2015
@nelsonic
Copy link
Member Author

nelsonic commented Dec 7, 2015

curl http://localhost:9200/twitter/tweet/_search?pretty=true -d '
{ 
    "query" : { 
        "match_all" : {} 
    },
    "fields": []
}
'

Gives:

{
  "took" : 4,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 56,
    "max_score" : 1.0,
    "hits" : [ {
      "_index" : "twitter",
      "_type" : "tweet",
      "_id" : "894117058953",
      "_score" : 1.0
    }, {
      "_index" : "twitter",
      "_type" : "tweet",
      "_id" : "559463492466782200",
      "_score" : 1.0
    }, {
      "_index" : "twitter",
      "_type" : "tweet",
      "_id" : "559463323897307140",
      "_score" : 1.0
    }, {
      "_index" : "twitter",
      "_type" : "tweet",
      "_id" : "559464056734896100",
      "_score" : 1.0
    }, {
      "_index" : "twitter",
      "_type" : "tweet",
      "_id" : "559464187764949000",
      "_score" : 1.0
    }, {
      "_index" : "twitter",
      "_type" : "tweet",
      "_id" : "559464082508496900",
      "_score" : 1.0
    }, {
      "_index" : "twitter",
      "_type" : "tweet",
      "_id" : "559463500398211100",
      "_score" : 1.0
    }, {
      "_index" : "twitter",
      "_type" : "tweet",
      "_id" : "559464234510471200",
      "_score" : 1.0
    }, {
      "_index" : "twitter",
      "_type" : "tweet",
      "_id" : "559464087571410940",
      "_score" : 1.0
    }, {
      "_index" : "twitter",
      "_type" : "tweet",
      "_id" : "559463492634562560",
      "_score" : 1.0
    } ]
  }
}

@nelsonic
Copy link
Member Author

nelsonic commented Dec 7, 2015

If we then combine this with a scroll: https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-scroll.html

curl http://localhost:9200/twitter/tweet/_search?scroll=5m&pretty=true -d '
{ 
    "query" : { 
        "match_all" : {} 
    },
    "fields": []
}
'

Returns:

{
  "_scroll_id": "cXVlcnlUaGVuRmV0Y2g7NTszNzE6UUgxVWhEYy1UZUNLTU10RWpPczdvUTszNzU6UUgxVWhEYy1UZUNLTU10RWpPczdvUTszNzI6UUgxVWhEYy1UZUNLTU10RWpPczdvUTszNzM6UUgxVWhEYy1UZUNLTU10RWpPczdvUTszNzQ6UUgxVWhEYy1UZUNLTU10RWpPczdvUTswOw==",
  "took": 2,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 56,
    "max_score": 1,
    "hits": [
      {
        "_index": "twitter",
        "_type": "tweet",
        "_id": "894117058953",
        "_score": 1,
        "_source": {
          "message": "My Random Message"
        }
      },
      {
        "_index": "twitter",
        "_type": "tweet",
        "_id": "559463492466782200",
        "_score": 1,
        "_source": {
          "created_at": "Sun Jan 25 21:31:05 +0000 2015",
          "id_str": "559463492466782208",
          "text": "Photo: \"Shuu acknowledges the amazing magnificence of Ken Kaneki. (Rate: 10 out of 10)\"\u00a0 SO. ACCURATE.\u00a0 http:\/\/t.co\/u0RF8t1oQy",
          "source": "<a href=\"http:\/\/www.tumblr.com\/\" rel=\"nofollow\">Tumblr<\/a>",
          "truncated": false,
          "user": {
            "id": 156514410,
            "id_str": "156514410",
            "name": "Kat.Sq",
            "screen_name": "KatDuterre",
            "location": "Canada",
            "url": "http:\/\/vkontakte.ru\/catrice",
            "description": null,
            "protected": false,
            "verified": false,
            "followers_count": 19,
            "friends_count": 88,
            "listed_count": 1,
            "favourites_count": 36,
            "statuses_count": 875,
            "created_at": "Thu Jun 17 04:07:44 +0000 2010",
            "utc_offset": -21600,
            "time_zone": "Central Time (US & Canada)",
            "geo_enabled": true,
            "lang": "ru",
            "contributors_enabled": false,
            "is_translator": false,
            "profile_background_color": "F5F4EB",
            "profile_background_image_url": "http:\/\/pbs.twimg.com\/profile_background_images\/707951865\/a7569f944abdcc19fdff4520170b33bc.jpeg",
            "profile_background_image_url_https": "https:\/\/pbs.twimg.com\/profile_background_images\/707951865\/a7569f944abdcc19fdff4520170b33bc.jpeg",
            "profile_background_tile": true,
            "profile_link_color": "12B31C",
            "profile_sidebar_border_color": "FFFFFF",
            "profile_sidebar_fill_color": "F6FFD1",
            "profile_text_color": "333333",
            "profile_use_background_image": true,
            "profile_image_url": "http:\/\/pbs.twimg.com\/profile_images\/3450194233\/fa8dcccce07a4f87b465af8366d72fe9_normal.jpeg",
            "profile_image_url_https": "https:\/\/pbs.twimg.com\/profile_images\/3450194233\/fa8dcccce07a4f87b465af8366d72fe9_normal.jpeg",
            "profile_banner_url": "https:\/\/pbs.twimg.com\/profile_banners\/156514410\/1364617701",
            "default_profile": false,
            "default_profile_image": false,
            "following": null,
            "follow_request_sent": null,
            "notifications": null
          },
          "retweet_count": 0,
          "favorite_count": 0,
          "entities": {
            "hashtags": [

            ],
            "trends": [

            ],
            "urls": [
              {
                "url": "http:\/\/t.co\/u0RF8t1oQy",
                "expanded_url": "http:\/\/tmblr.co\/ZNHhPx1bf8JiG",
                "display_url": "tmblr.co\/ZNHhPx1bf8JiG",
                "indices": [
                  104,
                  126
                ]
              }
            ],
            "user_mentions": [

            ],
            "symbols": [

            ]
          },
          "favorited": false,
          "retweeted": false,
          "possibly_sensitive": false,
          "filter_level": "low",
          "lang": "en",
          "timestamp_ms": "1422221465590"
        }
      },
      {
        "_index": "twitter",
        "_type": "tweet",
        "_id": "559463323897307140",
        "_score": 1,
        "_source": {
          "created_at": "Sun Jan 25 21:30:25 +0000 2015",
          "id_str": "559463323897307136",
          "text": "BuyNeu: Get or create amazing costumes of your favorite characters! http:\/\/t.co\/VAWQ7m8DNn",
          "source": "<a href=\"http:\/\/dlvr.it\" rel=\"nofollow\">dlvr.it<\/a>",
          "truncated": false,
          "user": {
            "id": 2202251011,
            "id_str": "2202251011",
            "name": "Lorretta Schrupp",
            "screen_name": "LorrettaSchrupp",
            "location": "",
            "url": "http:\/\/comic.stylinghub.com\/",
            "description": "I love comic, I even created a site just to share all about comics. #comics",
            "protected": false,
            "verified": false,
            "followers_count": 557,
            "friends_count": 11,
            "listed_count": 18,
            "favourites_count": 0,
            "statuses_count": 52095,
            "created_at": "Tue Nov 19 01:03:09 +0000 2013",
            "utc_offset": null,
            "time_zone": null,
            "geo_enabled": false,
            "lang": "en",
            "contributors_enabled": false,
            "is_translator": false,
            "profile_background_color": "0000FF",
            "profile_background_image_url": "http:\/\/pbs.twimg.com\/profile_background_images\/378800000123514465\/ace7ed8a6d9541fb171ecc4ece263a2d.jpeg",
            "profile_background_image_url_https": "https:\/\/pbs.twimg.com\/profile_background_images\/378800000123514465\/ace7ed8a6d9541fb171ecc4ece263a2d.jpeg",
            "profile_background_tile": true,
            "profile_link_color": "0000FF",
            "profile_sidebar_border_color": "C0DEED",
            "profile_sidebar_fill_color": "DDEEF6",
            "profile_text_color": "333333",
            "profile_use_background_image": true,
            "profile_image_url": "http:\/\/pbs.twimg.com\/profile_images\/378800000788897977\/96a737275686656256d76f828694ff11_normal.jpeg",
            "profile_image_url_https": "https:\/\/pbs.twimg.com\/profile_images\/378800000788897977\/96a737275686656256d76f828694ff11_normal.jpeg",
            "default_profile": false,
            "default_profile_image": false,
            "following": null,
            "follow_request_sent": null,
            "notifications": null
          },
          "retweet_count": 0,
          "favorite_count": 0,
          "entities": {
            "hashtags": [

            ],
            "trends": [

            ],
            "urls": [
              {
                "url": "http:\/\/t.co\/VAWQ7m8DNn",
                "expanded_url": "http:\/\/dlvr.it\/8DzdJ9",
                "display_url": "dlvr.it\/8DzdJ9",
                "indices": [
                  68,
                  90
                ]
              }
            ],
            "user_mentions": [

            ],
            "symbols": [

            ]
          },
          "favorited": false,
          "retweeted": false,
          "possibly_sensitive": false,
          "filter_level": "low",
          "lang": "en",
          "timestamp_ms": "1422221425400"
        }
      },
      {
        "_index": "twitter",
        "_type": "tweet",
        "_id": "559464056734896100",
        "_score": 1,
        "_source": {
          "created_at": "Sun Jan 25 21:33:20 +0000 2015",
          "id_str": "559464056734896128",
          "text": "Just turning off your phone for a while is an amazing thing",
          "source": "<a href=\"http:\/\/twitter.com\/download\/iphone\" rel=\"nofollow\">Twitter for iPhone<\/a>",
          "truncated": false,
          "user": {
            "id": 945003464,
            "id_str": "945003464",
            "name": "Kellianne",
            "screen_name": "kell_laurieee",
            "location": "",
            "url": null,
            "description": "\u029f\u026a\u0493\u1d07s \u026as \u029f\u026a\u1d0b\u1d07 \u1d00 \u1d04\u1d00\u1d0d\u1d07\u0280\u1d00, \u0493\u1d0f\u1d04\u1d1cs \u1d0f\u0274 \u1d21\u029c\u1d00\u1d1b \u026as \u026a\u1d0d\u1d18\u1d0f\u1d1b\u1d00\u0274\u1d1b & \u028f\u1d0f\u1d1c'\u029f\u029f \u1d04\u1d00\u1d18\u1d1b\u1d1c\u0280\u1d07 \u026a\u1d1b \u1d18\u1d07\u0280\u0493\u1d07\u1d04\u1d1b\u029f\u028f\u2661 \u0492\u03b1\u01a8\u043d\u03b9\u03c3\u0438\u2661D\u0280\u03b5\u03b1\u028d",
            "protected": false,
            "verified": false,
            "followers_count": 506,
            "friends_count": 615,
            "listed_count": 3,
            "favourites_count": 12817,
            "statuses_count": 5175,
            "created_at": "Tue Nov 13 02:08:29 +0000 2012",
            "utc_offset": -18000,
            "time_zone": "Eastern Time (US & Canada)",
            "geo_enabled": false,
            "lang": "en",
            "contributors_enabled": false,
            "is_translator": false,
            "profile_background_color": "642D8B",
            "profile_background_image_url": "http:\/\/pbs.twimg.com\/profile_background_images\/721240489\/36f937d9f37c3502522611bdfe5b452f.jpeg",
            "profile_background_image_url_https": "https:\/\/pbs.twimg.com\/profile_background_images\/721240489\/36f937d9f37c3502522611bdfe5b452f.jpeg",
            "profile_background_tile": true,
            "profile_link_color": "FF0000",
            "profile_sidebar_border_color": "FFFFFF",
            "profile_sidebar_fill_color": "7AC3EE",
            "profile_text_color": "3D1957",
            "profile_use_background_image": true,
            "profile_image_url": "http:\/\/pbs.twimg.com\/profile_images\/557020799564263428\/YVHg87d-_normal.jpeg",
            "profile_image_url_https": "https:\/\/pbs.twimg.com\/profile_images\/557020799564263428\/YVHg87d-_normal.jpeg",
            "profile_banner_url": "https:\/\/pbs.twimg.com\/profile_banners\/945003464\/1420600283",
            "default_profile": false,
            "default_profile_image": false,
            "following": null,
            "follow_request_sent": null,
            "notifications": null
          },
          "retweet_count": 0,
          "favorite_count": 0,
          "entities": {
            "hashtags": [

            ],
            "trends": [

            ],
            "urls": [

            ],
            "user_mentions": [

            ],
            "symbols": [

            ]
          },
          "favorited": false,
          "retweeted": false,
          "possibly_sensitive": false,
          "filter_level": "low",
          "lang": "en",
          "timestamp_ms": "1422221600122"
        }
      },
      {
        "_index": "twitter",
        "_type": "tweet",
        "_id": "559464187764949000",
        "_score": 1,
        "_source": {
          "created_at": "Sun Jan 25 21:33:51 +0000 2015",
          "id_str": "559464187764948993",
          "text": "\u2665 @camerondallas, I hope you're having a good day baby! if u're reading this I want you to smile\u30c4 bc you are amazing &amp; I love you \u2665 x519",
          "source": "<a href=\"http:\/\/twitter.com\" rel=\"nofollow\">Twitter Web Client<\/a>",
          "truncated": false,
          "user": {
            "id": 865191630,
            "id_str": "865191630",
            "name": "\u2727\u2218* Cameron\u2727\u2218",
            "screen_name": "oh5SOSorgasms",
            "location": "Neverland",
            "url": null,
            "description": "\u275dand i'm thinking about how people fall in love in mysterious ways\u275e1\/midnight red + colton follows",
            "protected": false,
            "verified": false,
            "followers_count": 833,
            "friends_count": 1425,
            "listed_count": 2,
            "favourites_count": 252,
            "statuses_count": 2614,
            "created_at": "Sat Oct 06 19:37:49 +0000 2012",
            "utc_offset": 3600,
            "time_zone": "Madrid",
            "geo_enabled": false,
            "lang": "es",
            "contributors_enabled": false,
            "is_translator": false,
            "profile_background_color": "C0DEED",
            "profile_background_image_url": "http:\/\/pbs.twimg.com\/profile_background_images\/556601770495848448\/bSWqQCBf.png",
            "profile_background_image_url_https": "https:\/\/pbs.twimg.com\/profile_background_images\/556601770495848448\/bSWqQCBf.png",
            "profile_background_tile": true,
            "profile_link_color": "0084B4",
            "profile_sidebar_border_color": "000000",
            "profile_sidebar_fill_color": "DDEEF6",
            "profile_text_color": "333333",
            "profile_use_background_image": true,
            "profile_image_url": "http:\/\/pbs.twimg.com\/profile_images\/557980762171924481\/PeOlDETX_normal.jpeg",
            "profile_image_url_https": "https:\/\/pbs.twimg.com\/profile_images\/557980762171924481\/PeOlDETX_normal.jpeg",
            "profile_banner_url": "https:\/\/pbs.twimg.com\/profile_banners\/865191630\/1421868541",
            "default_profile": false,
            "default_profile_image": false,
            "following": null,
            "follow_request_sent": null,
            "notifications": null
          },
          "retweet_count": 0,
          "favorite_count": 0,
          "entities": {
            "hashtags": [

            ],
            "trends": [

            ],
            "urls": [

            ],
            "user_mentions": [
              {
                "screen_name": "camerondallas",
                "name": "Cameron Dallas",
                "id": 405728790,
                "id_str": "405728790",
                "indices": [
                  2,
                  16
                ]
              }
            ],
            "symbols": [

            ]
          },
          "favorited": false,
          "retweeted": false,
          "possibly_sensitive": false,
          "filter_level": "low",
          "lang": "en",
          "timestamp_ms": "1422221631362"
        }
      },
      {
        "_index": "twitter",
        "_type": "tweet",
        "_id": "559464082508496900",
        "_score": 1,
        "_source": {
          "created_at": "Sun Jan 25 21:33:26 +0000 2015",
          "id_str": "559464082508496896",
          "text": "Just thought of something potentially amazing. @berner415 . how about some marketing with Brown's WR josh Gordon?",
          "source": "<a href=\"http:\/\/twitter.com\/download\/android\" rel=\"nofollow\">Twitter for Android<\/a>",
          "truncated": false,
          "user": {
            "id": 2490411656,
            "id_str": "2490411656",
            "name": "Harold ",
            "screen_name": "TrvpHvrold",
            "location": "San Diego",
            "url": null,
            "description": "Just an 18 year old chillin playing league and listening to music. #tgod #uzi #hbk",
            "protected": false,
            "verified": false,
            "followers_count": 79,
            "friends_count": 223,
            "listed_count": 0,
            "favourites_count": 4958,
            "statuses_count": 1353,
            "created_at": "Sun May 11 21:15:52 +0000 2014",
            "utc_offset": -25200,
            "time_zone": "Arizona",
            "geo_enabled": false,
            "lang": "en",
            "contributors_enabled": false,
            "is_translator": false,
            "profile_background_color": "1A1B1F",
            "profile_background_image_url": "http:\/\/pbs.twimg.com\/profile_background_images\/552664587917459456\/j_U-qq02.png",
            "profile_background_image_url_https": "https:\/\/pbs.twimg.com\/profile_background_images\/552664587917459456\/j_U-qq02.png",
            "profile_background_tile": true,
            "profile_link_color": "2FC2EF",
            "profile_sidebar_border_color": "FFFFFF",
            "profile_sidebar_fill_color": "252429",
            "profile_text_color": "666666",
            "profile_use_background_image": true,
            "profile_image_url": "http:\/\/pbs.twimg.com\/profile_images\/470662702394642432\/Pd37cmW7_normal.jpeg",
            "profile_image_url_https": "https:\/\/pbs.twimg.com\/profile_images\/470662702394642432\/Pd37cmW7_normal.jpeg",
            "profile_banner_url": "https:\/\/pbs.twimg.com\/profile_banners\/2490411656\/1421029192",
            "default_profile": false,
            "default_profile_image": false,
            "following": null,
            "follow_request_sent": null,
            "notifications": null
          },
          "retweet_count": 0,
          "favorite_count": 0,
          "entities": {
            "hashtags": [

            ],
            "trends": [

            ],
            "urls": [

            ],
            "user_mentions": [
              {
                "screen_name": "berner415",
                "name": "#20LIGHTS",
                "id": 22312032,
                "id_str": "22312032",
                "indices": [
                  47,
                  57
                ]
              }
            ],
            "symbols": [

            ]
          },
          "favorited": false,
          "retweeted": false,
          "possibly_sensitive": false,
          "filter_level": "low",
          "lang": "en",
          "timestamp_ms": "1422221606267"
        }
      },
      {
        "_index": "twitter",
        "_type": "tweet",
        "_id": "559463500398211100",
        "_score": 1,
        "_source": {
          "created_at": "Sun Jan 25 21:31:07 +0000 2015",
          "id_str": "559463500398211072",
          "text": "Hi @Harry_Styles \nYou're amazing\ntalented\nthoughtful\ncaring\nkind\nthe list is endless\u30c4\nHope that made you smile\nfollow me? \nI love u x30,828",
          "source": "<a href=\"http:\/\/twitter.com\/#!\/download\/ipad\" rel=\"nofollow\">Twitter for iPad<\/a>",
          "truncated": false,
          "user": {
            "id": 1872033427,
            "id_str": "1872033427",
            "name": "lotte loves harry",
            "screen_name": "harrehstrokes",
            "location": "cheshire ",
            "url": "https:\/\/twitter.com\/harrehstrokes\/status\/549266660537151488",
            "description": "it costs nothing to be nice",
            "protected": false,
            "verified": false,
            "followers_count": 1301,
            "friends_count": 962,
            "listed_count": 45,
            "favourites_count": 5855,
            "statuses_count": 60397,
            "created_at": "Mon Sep 16 15:13:40 +0000 2013",
            "utc_offset": 3600,
            "time_zone": "Amsterdam",
            "geo_enabled": true,
            "lang": "en",
            "contributors_enabled": false,
            "is_translator": false,
            "profile_background_color": "C0DEED",
            "profile_background_image_url": "http:\/\/abs.twimg.com\/images\/themes\/theme1\/bg.png",
            "profile_background_image_url_https": "https:\/\/abs.twimg.com\/images\/themes\/theme1\/bg.png",
            "profile_background_tile": false,
            "profile_link_color": "0084B4",
            "profile_sidebar_border_color": "C0DEED",
            "profile_sidebar_fill_color": "DDEEF6",
            "profile_text_color": "333333",
            "profile_use_background_image": true,
            "profile_image_url": "http:\/\/pbs.twimg.com\/profile_images\/554322183779282945\/3ZfS1Oai_normal.jpeg",
            "profile_image_url_https": "https:\/\/pbs.twimg.com\/profile_images\/554322183779282945\/3ZfS1Oai_normal.jpeg",
            "profile_banner_url": "https:\/\/pbs.twimg.com\/profile_banners\/1872033427\/1420995683",
            "default_profile": true,
            "default_profile_image": false,
            "following": null,
            "follow_request_sent": null,
            "notifications": null
          },
          "retweet_count": 0,
          "favorite_count": 0,
          "entities": {
            "hashtags": [

            ],
            "trends": [

            ],
            "urls": [

            ],
            "user_mentions": [
              {
                "screen_name": "Harry_Styles",
                "name": "Harry Styles.",
                "id": 181561712,
                "id_str": "181561712",
                "indices": [
                  3,
                  16
                ]
              }
            ],
            "symbols": [

            ]
          },
          "favorited": false,
          "retweeted": false,
          "possibly_sensitive": false,
          "filter_level": "low",
          "lang": "en",
          "timestamp_ms": "1422221467481"
        }
      },
      {
        "_index": "twitter",
        "_type": "tweet",
        "_id": "559464234510471200",
        "_score": 1,
        "_source": {
          "created_at": "Sun Jan 25 21:34:02 +0000 2015",
          "id_str": "559464234510471168",
          "text": "@rcollete I see the dresses amazing collection! Nices shots! Versace knows how makes women godness. Thanks for that.",
          "source": "<a href=\"http:\/\/twitter.com\/download\/android\" rel=\"nofollow\">Twitter for Android<\/a>",
          "truncated": false,
          "in_reply_to_status_id": 5.5946160032816e+17,
          "in_reply_to_status_id_str": "559461600328155136",
          "in_reply_to_user_id": 86361893,
          "in_reply_to_user_id_str": "86361893",
          "in_reply_to_screen_name": "rcollete",
          "user": {
            "id": 59805224,
            "id_str": "59805224",
            "name": "Jana\u00edra Fran\u00e7a",
            "screen_name": "JanairaFranca",
            "location": "Brazil",
            "url": null,
            "description": "Mulher imperfeita, real e curiosa #Marketing #DigitalMarketing #Communication #SocialMedia \u2605\u2605\u2605\u2605\u2605 S\u00e3o Paulo",
            "protected": false,
            "verified": false,
            "followers_count": 570,
            "friends_count": 401,
            "listed_count": 11,
            "favourites_count": 757,
            "statuses_count": 5783,
            "created_at": "Fri Jul 24 15:10:43 +0000 2009",
            "utc_offset": -7200,
            "time_zone": "Brasilia",
            "geo_enabled": true,
            "lang": "pt",
            "contributors_enabled": false,
            "is_translator": false,
            "profile_background_color": "F2F7F7",
            "profile_background_image_url": "http:\/\/pbs.twimg.com\/profile_background_images\/534769366550605824\/nM8g64gQ.jpeg",
            "profile_background_image_url_https": "https:\/\/pbs.twimg.com\/profile_background_images\/534769366550605824\/nM8g64gQ.jpeg",
            "profile_background_tile": false,
            "profile_link_color": "0725BA",
            "profile_sidebar_border_color": "FFFFFF",
            "profile_sidebar_fill_color": "DDFFCC",
            "profile_text_color": "333333",
            "profile_use_background_image": false,
            "profile_image_url": "http:\/\/pbs.twimg.com\/profile_images\/559458053834428418\/KTtYakOD_normal.jpeg",
            "profile_image_url_https": "https:\/\/pbs.twimg.com\/profile_images\/559458053834428418\/KTtYakOD_normal.jpeg",
            "profile_banner_url": "https:\/\/pbs.twimg.com\/profile_banners\/59805224\/1416334434",
            "default_profile": false,
            "default_profile_image": false,
            "following": null,
            "follow_request_sent": null,
            "notifications": null
          },
          "retweet_count": 0,
          "favorite_count": 0,
          "entities": {
            "hashtags": [

            ],
            "trends": [

            ],
            "urls": [

            ],
            "user_mentions": [
              {
                "screen_name": "rcollete",
                "name": "Collete, Ralf A.L.L ",
                "id": 86361893,
                "id_str": "86361893",
                "indices": [
                  0,
                  9
                ]
              }
            ],
            "symbols": [

            ]
          },
          "favorited": false,
          "retweeted": false,
          "possibly_sensitive": false,
          "filter_level": "low",
          "lang": "en",
          "timestamp_ms": "1422221642507"
        }
      },
      {
        "_index": "twitter",
        "_type": "tweet",
        "_id": "559464087571410940",
        "_score": 1,
        "_source": {
          "created_at": "Sun Jan 25 21:33:27 +0000 2015",
          "id_str": "559464087571410945",
          "text": "@jg_28 thanks Jackie for donating, the support has been amazing\ud83d\ude0a",
          "source": "<a href=\"http:\/\/twitter.com\/download\/iphone\" rel=\"nofollow\">Twitter for iPhone<\/a>",
          "truncated": false,
          "in_reply_to_status_id": 5.5945990366089e+17,
          "in_reply_to_status_id_str": "559459903660892160",
          "in_reply_to_user_id": 297074652,
          "in_reply_to_user_id_str": "297074652",
          "in_reply_to_screen_name": "jg_28",
          "user": {
            "id": 701721223,
            "id_str": "701721223",
            "name": "Vic Kane",
            "screen_name": "VicGarbutt",
            "location": "Newtownabbey, Northern Ireland",
            "url": null,
            "description": "Life's for livin, laughin & lovin. Mummy to gorgeous sons Archie & Teddie, love my job in a boys school, live for fashion & food!",
            "protected": false,
            "verified": false,
            "followers_count": 346,
            "friends_count": 986,
            "listed_count": 4,
            "favourites_count": 179,
            "statuses_count": 292,
            "created_at": "Tue Jul 17 20:27:47 +0000 2012",
            "utc_offset": null,
            "time_zone": null,
            "geo_enabled": true,
            "lang": "en",
            "contributors_enabled": false,
            "is_translator": false,
            "profile_background_color": "C0DEED",
            "profile_background_image_url": "http:\/\/abs.twimg.com\/images\/themes\/theme1\/bg.png",
            "profile_background_image_url_https": "https:\/\/abs.twimg.com\/images\/themes\/theme1\/bg.png",
            "profile_background_tile": false,
            "profile_link_color": "0084B4",
            "profile_sidebar_border_color": "C0DEED",
            "profile_sidebar_fill_color": "DDEEF6",
            "profile_text_color": "333333",
            "profile_use_background_image": true,
            "profile_image_url": "http:\/\/pbs.twimg.com\/profile_images\/489867527762489345\/DmWV7B94_normal.jpeg",
            "profile_image_url_https": "https:\/\/pbs.twimg.com\/profile_images\/489867527762489345\/DmWV7B94_normal.jpeg",
            "profile_banner_url": "https:\/\/pbs.twimg.com\/profile_banners\/701721223\/1405685799",
            "default_profile": true,
            "default_profile_image": false,
            "following": null,
            "follow_request_sent": null,
            "notifications": null
          },
          "retweet_count": 0,
          "favorite_count": 0,
          "entities": {
            "hashtags": [

            ],
            "trends": [

            ],
            "urls": [

            ],
            "user_mentions": [
              {
                "screen_name": "jg_28",
                "name": "Jack Gourley",
                "id": 297074652,
                "id_str": "297074652",
                "indices": [
                  0,
                  6
                ]
              }
            ],
            "symbols": [

            ]
          },
          "favorited": false,
          "retweeted": false,
          "possibly_sensitive": false,
          "filter_level": "low",
          "lang": "en",
          "timestamp_ms": "1422221607474"
        }
      },
      {
        "_index": "twitter",
        "_type": "tweet",
        "_id": "559463492634562560",
        "_score": 1,
        "_source": {
          "created_at": "Sun Jan 25 21:31:05 +0000 2015",
          "id_str": "559463492634562561",
          "text": "Great job by Pastor Buddy and Alicia sharing about the amazing things God is doing at the Abba Center! http:\/\/t.co\/VU8SdINxvc",
          "source": "<a href=\"http:\/\/www.facebook.com\/twitter\" rel=\"nofollow\">Facebook<\/a>",
          "truncated": false,
          "user": {
            "id": 965649199,
            "id_str": "965649199",
            "name": "Core Church",
            "screen_name": "CoreChurchAC",
            "location": "Broken Arrow, OK",
            "url": "http:\/\/corechurch.com",
            "description": "At the core of who we are is loving God and loving people.",
            "protected": false,
            "verified": false,
            "followers_count": 37,
            "friends_count": 0,
            "listed_count": 0,
            "favourites_count": 0,
            "statuses_count": 593,
            "created_at": "Fri Nov 23 07:09:56 +0000 2012",
            "utc_offset": -18000,
            "time_zone": "Eastern Time (US & Canada)",
            "geo_enabled": false,
            "lang": "en",
            "contributors_enabled": false,
            "is_translator": false,
            "profile_background_color": "FAFDFF",
            "profile_background_image_url": "http:\/\/pbs.twimg.com\/profile_background_images\/378800000084262591\/194993eca6ac03b108b1a6e03659b703.jpeg",
            "profile_background_image_url_https": "https:\/\/pbs.twimg.com\/profile_background_images\/378800000084262591\/194993eca6ac03b108b1a6e03659b703.jpeg",
            "profile_background_tile": true,
            "profile_link_color": "CF9D06",
            "profile_sidebar_border_color": "000000",
            "profile_sidebar_fill_color": "EFEFEF",
            "profile_text_color": "333333",
            "profile_use_background_image": true,
            "profile_image_url": "http:\/\/pbs.twimg.com\/profile_images\/2884638944\/8985447c0a3d4ad207639076ad141ae3_normal.jpeg",
            "profile_image_url_https": "https:\/\/pbs.twimg.com\/profile_images\/2884638944\/8985447c0a3d4ad207639076ad141ae3_normal.jpeg",
            "default_profile": false,
            "default_profile_image": false,
            "following": null,
            "follow_request_sent": null,
            "notifications": null
          },
          "retweet_count": 0,
          "favorite_count": 0,
          "entities": {
            "hashtags": [

            ],
            "trends": [

            ],
            "urls": [
              {
                "url": "http:\/\/t.co\/VU8SdINxvc",
                "expanded_url": "http:\/\/fb.me\/1Uqr39fwX",
                "display_url": "fb.me\/1Uqr39fwX",
                "indices": [
                  103,
                  125
                ]
              }
            ],
            "user_mentions": [

            ],
            "symbols": [

            ]
          },
          "favorited": false,
          "retweeted": false,
          "possibly_sensitive": false,
          "filter_level": "low",
          "lang": "en",
          "timestamp_ms": "1422221465630"
        }
      }
    ]
  }
}

@nelsonic
Copy link
Member Author

nelsonic commented Dec 7, 2015

Apparently we're going to need to use from/size to increase the number of hits above 10:
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-from-size.html

@raed667
Copy link
Contributor

raed667 commented Dec 22, 2016

@nelsonic : Does the client SEARCH support size and from parameters ?

@nelsonic
Copy link
Member Author

@RaedsLab not yet... if you have time to add, please go for it! 👍
(P.S: I can make you the maintainer if you're interested...)
Only two of our projects are using ElasticSearch right now (the rest are all PostgreSQL) so I'm a bit out of the game with ES.

CC: @SimonLab

@raed667
Copy link
Contributor

raed667 commented Dec 23, 2016

@nelsonic I have already submitted a PR #106 for this feature, which also closes #91

I have just started learning ES (3 days now 😄 ) so I might not be the most qualified person to maintain a client.

@nelsonic
Copy link
Member Author

@RaedsLab thank you so much for the Pull Request!! 🎉
I cannot tell you how excited we are to see people discovering our repos/projects and finding/making the time to improve them! 😍

(perhaps counterintuitively) the fact that you are a "beginner" (at ElasticSearch) makes you the perfect person to read through, extend and maintain this package! You are only "new" to a subject once. Having a fresh perspective means you can continuously ask the question:
"did I understand this the first time I saw it?" and if not, then an improvement is required.
in which case a new issue is created in the "backlog" for why it was not clear.

See: https://en.wikipedia.org/wiki/Shoshin (Beginner's Mind. what we aspire to...!)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants