Skip to content

Can't stop circular dataflow blocks #612

@ZhaoXiangXML

Description

@ZhaoXiangXML

I'm trying to implement a crawler with System.Threading.Tasks.Dataflow, the core feature is to have a TransformBlock, which accepts an url as input, request the url, parse the response, and find more urls for the same block.

Then I noticed a problem that I cannot complete/stop this pipeline. The following code simulates my problem:

using System;
using System.Threading;
using System.Threading.Tasks.Dataflow;

namespace ConsoleApp1
{
    class Program
    {
        static void Main()
        {
            int count = 0;

            var block = new TransformBlock<string, string>(uri =>
            {
                //await Task.Delay(10);
                Console.WriteLine($"Downloading {count}");

                if (Interlocked.Increment(ref count) > 10)
                {
                    return null;
                }

                return uri;
            });

            var linkOptions = new DataflowLinkOptions { PropagateCompletion = false };

            block.LinkTo(block, linkOptions, (x) => !string.IsNullOrEmpty(x));

            block.Post("");
            block.Post("");

            while (true)
            {
                Console.WriteLine($"Check Count: {count}");

                if (count >= 2)
                {
                    Console.WriteLine("Complete");
                    block.Complete();
                    break;
                }

                Thread.Sleep(20);
            }

            Console.WriteLine("After Check");

            block.Completion.Wait();

            Console.WriteLine("Done");
        }
    }
}

The output is:

Check Count: 0
Downloading 0
Downloading 1
Check Count: 2
Complete
After Check

And program stuck here, block.Completion.Wait() never return.

I've also tried to have two blocks link to each other, yet the result is same.

So what can I do about my code? How can I stop this dataflow?

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions