[core] Avoid blocking in Thread<Object> constructor (#12151)

* Introduce AspiringActor, EstablishedActor

This pair of objects represents the two-phase (parent-thread /
child-thread) construction that's needed to support constructing
Thread<Object> without blocking until the child thread is up and
running.

An `AspiringActor<O>` is responsible for:
 - ownership of the actor's `Mailbox`
 - allocating the memory for (but *not* constructing) the target object `O`

Using these two pieces--the mailbox and a stable address for `O`--an
`AspiringActor<O>` can accept messages for the target object, or provide
`ActorRef<O>`s that do so, before the object has actually been
constructed by the corresponding `EstablishedActor<O>`.  (Such messages
are queued in the mailbox until after the object is constructed.)

This allows for an `AspiringActor<O>` to be created and safely used by a
thread other than the one on which the target object will (eventually)
live.

An `EstablishedActor<O>` is responsible for managing the lifetime of the
target object `O` and the open/closed state of the parent's `mailbox`.

The `O` object's lifetime is contained by that of its owning
`EstablishedActor<O>`: the `EstablishedActor` constructor executes the
`O` constructor via "placement new", constructing it at the address
provided by the parent `AspiringActor`, and the `~EstablishedActor`
destructor similarly executes the `~O` destructor (after closing the
mailbox). `EstablishedActor` should therefore live entirely on the
thread intended to own `O`.

* Remove Actor#{invoke,ask}

1 files changed, 59 insertions, 2 deletions

diff --git a/test/util/thread.test.cpp b/test/util/thread.test.cpp
index 76fb5ce3f0..2bcb9d8959 100644
--- a/test/util/thread.test.cpp
+++ b/test/util/thread.test.cpp
@@ -15,11 +15,11 @@ class TestObject {
 public:
     TestObject(ActorRef<TestObject>, std::thread::id otherTid)
         : tid(std::this_thread::get_id()) {
-        EXPECT_NE(tid, otherTid);
+        EXPECT_NE(tid, otherTid); // Object is created on child thread
     }
 
     ~TestObject() {
-        EXPECT_EQ(tid, std::this_thread::get_id());
+        EXPECT_EQ(tid, std::this_thread::get_id()); // Object is destroyed on child thread
     }
 
     void fn1(int val) const {
@@ -275,3 +275,60 @@ TEST(Thread, PauseResume) {
     thread.actor().invoke(&TestWorker::send, [&] { loop.stop(); });
     loop.run();
 }
+
+
+class TestWorkerDelayedConstruction {
+public:
+    TestWorkerDelayedConstruction(ActorRef<TestWorkerDelayedConstruction>, std::future<void> start) {
+        start.get();
+    }
+
+    void send(std::function<void ()> cb) {
+        cb();
+    }
+
+private:
+    Timer timer;
+};
+
+TEST(Thread, InvokeBeforeChildStarts) {
+    RunLoop loop;
+
+    std::promise<void> start;
+    Thread<TestWorkerDelayedConstruction> thread("Test", start.get_future());
+    
+    std::atomic<int> count { 0 };
+    
+    for (unsigned i = 0; i < 100; ++i) {
+        thread.actor().invoke(&TestWorkerDelayedConstruction::send, [&] { ++count; });
+    }
+
+    thread.actor().invoke(&TestWorkerDelayedConstruction::send, [&] { loop.stop(); });
+
+    // This test will be flaky if messages are consumed before the target object is constructed.
+    ASSERT_EQ(count, 0);
+
+    start.set_value();
+    
+    loop.run();
+
+    ASSERT_EQ(count, 100);
+}
+
+TEST(Thread, DeleteBeforeChildStarts) {
+    std::atomic_bool flag(false);
+    std::promise<void> start;
+
+    Thread<TestWorker> control("Control");
+    auto thread = std::make_unique<Thread<TestWorkerDelayedConstruction>>("Test", start.get_future());
+    
+    thread->actor().invoke(&TestWorkerDelayedConstruction::send, [&] { flag = true; });
+
+    control.actor().invoke(&TestWorker::sendDelayed, [&] { start.set_value(); });
+
+    // Should not hang.
+    thread.reset();
+
+    // Should process the queue before destruction.
+    ASSERT_TRUE(flag);
+}