Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chunk with 0 messages #438

Open
bilogic opened this issue May 9, 2022 · 1 comment
Open

Chunk with 0 messages #438

bilogic opened this issue May 9, 2022 · 1 comment

Comments

@bilogic
Copy link

bilogic commented May 9, 2022

Describe the bug
Chunk returns with 0 messages when there are still >1000 messages on server

To Reproduce
Steps to reproduce the behavior:
Connecting to gmail, $chunk_size and $start_chunk set to 1
The issue remains even if $chunk_size is set to 10

$i  = 0;
$folder->query()->all()->chunked(function ($messages, $chunk) use (&$i) {
    echo "- chunk #$chunk, ".$messages->count().' messages';
    $messages->each(function (Message $message) use (&$i) {
        file_put_contents(storage_path("messages/$i.xml"), $message->getRawBody());
        $i++;

        if ($message->move('INBOX/Downloaded') == true) {
            echo "- Moved {$message->uid}").PHP_EOL;
        } else {
            echo "- Cannot move {$message->uid}".PHP_EOL;
        }
    });
}, $chunk_size = 1, $start_chunk = 1);

Expected behavior
There should always be >0 messages in each chunk, never 0 messages.

Screenshots

- chunk #1, 1 messages
- Moved 6003
- chunk #2, 1 messages
- Moved 11
- chunk #3, 1 messages
- Moved 20
- chunk #4, 1 messages
- Moved 27
- chunk #5, 1 messages
- Moved 31
- chunk #6, 1 messages
- Moved 35
- chunk #7, 1 messages
- Moved 38
- chunk #8, 1 messages
- Moved 40
- chunk #9, 1 messages
- Moved 42
- chunk #10, 1 messages
- Moved 44
- chunk #11, 1 messages
- Moved 46
- chunk #12, 1 messages
- Moved 48
- chunk #13, 0 messages
- chunk #14, 0 messages
- chunk #15, 0 messages
- chunk #16, 0 messages
- chunk #17, 0 messages
- chunk #18, 0 messages
- chunk #19, 0 messages
- chunk #20, 0 messages

Desktop / Server (please complete the following information):

  • PHP version: 7.4.28
  • Laravel version: 8.83.10
  • Nova version: 3.32.0
  • MySQL version: 8.0.29-0ubuntu0.21.10.2
  • MySQL Client version: mysqlnd 7.4.28
  • databaseEngine version: mysql
  • databaseVersion2 version: 8.0.29-0ubuntu0.21.10.2

Additional context

  1. I want to download and save raw messages one by one without it becoming a memory hog.
  2. Once I kill the script and restart, it downloads another 12 to 13 messages before returning 0 messages again, so it doesn't seem to be a rate limiting issue
@Tahiaji
Copy link

Tahiaji commented Dec 8, 2022

The problem is: when you move email - first item from collection removed too. Its like iterate array and remove elements.

Workaround for me - iterate 1 by 1, and request every time first element

/** @var WhereQuery $query */
$totalMessages = $this->inboxFolder->messages()->all()->count();
for ($i = 1; $i <= $totalMessages; $i++) {
   /** @var MessageCollection $messages */
   $messages = $this->inboxFolder->messages()->all()->limit(1, 1)->get();
   foreach ($messages as $message) {
      $message->move('...'); // and so on
   } 
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants