Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't stop circular dataflow blocks #612

Closed
ZhaoXiangXML opened this issue Dec 6, 2019 · 3 comments
Closed

Can't stop circular dataflow blocks #612

ZhaoXiangXML opened this issue Dec 6, 2019 · 3 comments

Comments

@ZhaoXiangXML
Copy link

I'm trying to implement a crawler with System.Threading.Tasks.Dataflow, the core feature is to have a TransformBlock, which accepts an url as input, request the url, parse the response, and find more urls for the same block.

Then I noticed a problem that I cannot complete/stop this pipeline. The following code simulates my problem:

using System;
using System.Threading;
using System.Threading.Tasks.Dataflow;

namespace ConsoleApp1
{
    class Program
    {
        static void Main()
        {
            int count = 0;

            var block = new TransformBlock<string, string>(uri =>
            {
                //await Task.Delay(10);
                Console.WriteLine($"Downloading {count}");

                if (Interlocked.Increment(ref count) > 10)
                {
                    return null;
                }

                return uri;
            });

            var linkOptions = new DataflowLinkOptions { PropagateCompletion = false };

            block.LinkTo(block, linkOptions, (x) => !string.IsNullOrEmpty(x));

            block.Post("");
            block.Post("");

            while (true)
            {
                Console.WriteLine($"Check Count: {count}");

                if (count >= 2)
                {
                    Console.WriteLine("Complete");
                    block.Complete();
                    break;
                }

                Thread.Sleep(20);
            }

            Console.WriteLine("After Check");

            block.Completion.Wait();

            Console.WriteLine("Done");
        }
    }
}

The output is:

Check Count: 0
Downloading 0
Downloading 1
Check Count: 2
Complete
After Check

And program stuck here, block.Completion.Wait() never return.

I've also tried to have two blocks link to each other, yet the result is same.

So what can I do about my code? How can I stop this dataflow?

@Dotnet-GitSync-Bot Dotnet-GitSync-Bot added area-System.Threading untriaged New issue has not been triaged by the area owner labels Dec 6, 2019
@ZhaoXiangXML
Copy link
Author

Versions of things: Windows 10.0.17763.864 64bit, .NET Core 3.0, System.Threading.Tasks.Dataflow 4.11.0

@ZhaoXiangXML
Copy link
Author

And I've also tried PropagateCompletion = true, the result is all same

@ZhaoXiangXML
Copy link
Author

Well problem solved by adding block.LinkTo(DataflowBlock.NullTarget<string>(), linkOptions);, thanks to the help of the sample code

@ghost ghost locked as resolved and limited conversation to collaborators Dec 11, 2020
@tannergooding tannergooding removed the untriaged New issue has not been triaged by the area owner label Jun 24, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

4 participants