I have spent several years on Node.js and libuv. I felt desperately that callback functions, Promise and async/await in Node.js work pretty well all the time. The callback functions is responsible for the low-level I/O, the data are read and written chunk by chunk. The Promise is responsible to wrap multiple callback functions to make a atomic I/O operation. And the async/await is responsible only to form a logical workflow.

And I have two to three years experience in asyncdispatch, asyncnet, asynchttpserver in Nim and I have written several asynchronous modules. Now, I'm writting an asynchronous Mysql client/connector in pure Nim. However, I now feel that it is not appropriate to use async/await entirely.

Now, let us suppose that we use a mysql client to execute some queries. This is a pseudo code:

var conn = newAsyncSocket()

proc query(sql: string): Stream {.async.} =
  await conn.send(mysql_format_function(sql))
  let data = await conn.recv_mysql_function()
  mysql_parse_function(data) 
  return newStream(conn)

proc main() {.async.} =
  let stream1 = await query("select 100; select 200;")
  while true:
    let data = await stream1.read()
    echo data  # 100, 200
    if data == "":
      break

  let stream2 = await query("select 300;  select 400;")
  while true:
    let data = await stream2.read()
    echo data  # 300, 400
    if data == "":
      break

waitFor main()

The code work fine if stream2 is always executing after stream1. However, if not:

proc main() {.async.} =
  let stream1 = await query("select 100; select 200;")
  let stream2 = await query("select 300;  select 400;")

  while true:
    let data = await stream1.read()
    echo data  # 100, 200
    if data == "":
      break

  while true:
    let data = await stream2.read()
    echo data  # 300, 400
    if data == "":
      break

or

proc do1() {.async.} =
  let stream1 = await query("select 100; select 200;")
  while true:
    let data = await stream1.read()
    echo data  # 100, 200
    if data == "":
      break

proc do2() {.async.} =
  while true:
    let data = await stream2.read()
    echo data  # 300, 400
    if data == "":
      break

proc main() {.async.} =
  asyncCheck do1()
  asyncCheck do2()

What happened then? The main function will blocking at (await) stream2 because stream1 is never finished. What does finished mean? The stream1 finished is mean that stream1 recv all the data that belong to the query of select 100; select 200;.

     stream1 data           stream2 data
|......................|....................|
|                      |                    |
v                      V                    V
stream1 begin       stream1 finished        stream2 finished
                    stream2 begin

A solution is to use lock or queue cache. This can solve the second problems, but it also introduce data race:

proc do1() {.async.} =
  await conn.lock()
  let stream1 = await query("select 100; select 200;")
  # ...
  conn.release()

proc do2() {.async.} =
  await conn.lock()
  let stream1 = await query("select 100; select 200;")
  # ...
  conn.release()

And it cannot prevent programmers to write the first kind of code:

proc main() {.async.} =
  let stream1 = await query("select 100; select 200;")
  let stream2 = await query("select 300;  select 400;")

Each of your asynchronous functions must be atomic. Otherwise, there are traps in them. The atomic function is mean that you can not construct streaming interfaces:

proc main() {.async.} =
  let all_data1 = await query("select 100; select 200;")
  let all_data2 = await query("select 300;  select 400;")

It is very bad to transfer large data, and it is a huge issue whenever you use async/await to write any program .

Ideally, using some callback functions to execute low-level I/O, using some Future to wrap these callback functions into atomic operations, and using async/await to construct the final logical workflow, then you can get the perfect solution:

# `Future` wrapper
proc query(sql: string, fileStream: FileStream): retFuture[void]  =  
  var retFuture = newFuture[void]()

  let stream = conn.send(mysql_format_function(sql))  # I/O interface

  stream.onBegin() do (data: string):                 # I/O interface
    fileStream.write(data)
 
  stream.onFinished() do ():                          # I/O interface
    complete(retFuture)

# logical workflow
proc main() {.async.} =
  await query("select 100; select 200;")  # write to file 
  await query("select 300; select 400;")  # write to file