curl/Multi 接口超时调度

用伸展树汇总多个传输的最近超时

每个 easy handle 可有多种内部期限,但全局伸展树只保留它最近的一项,为 Multi 接口汇总下一次唤醒时间。

c00a3021e调度与资源分配核验 2026-10-12
相关概念

工程问题#

连接建立、传输进度等环节各有定时需求。若每次轮询扫描全部传输及其所有期限,成本会随并发量上升;应用却只需要一个下一次调用 libcurl 的时间。

实际实现#

Curl_expire_set 更新传输内部按期限组织的定时项,更新的不是原首项且原首项仍保持最早时跳过全局树更新,其余情况重新登记。Curl_timeouts_next_ms 将最小键伸展到根;到期时 splaygetbest 取出最小节点,相同期限通过 same 链处理。Multi 随后跳过该传输已到期的内部项,把下一项重新入树并标记传输需要推进。

处理流程#

  1. 为 easy handle 设置或替换某个 expire_id 对应的期限
  2. 将每个 handle 的局部最小期限登记到全局伸展树
  3. 伸展最小键到根,向应用返回剩余等待时间
  4. 移除到期 handle,重新登记其下一期限并推进传输状态

源码片段#

固定版本 · 原文摘录
lib/multi.cc00a3021e4b833afebed33f31a8f006a1b7511fe · L3775–L3794
查看原文件
3775  /* Add the timeout, will replace any previous value for this timer. */3776  multi_set_timeout(data, &set, eid);3777  DEBUGASSERT(timeouts->first < EXPIRE_LAST);37783779  if(Curl_timeouts_has(data)) {3780    /* data has already a timeout registered. If the first timer3781     * was NOT the one we just set AND is still the first one,3782     * nothing changed from the timeouts point of view. The3783     * set timer triggers after the one already registered. Leave. */3784    if((prev_id != eid) && (prev_id == timeouts->first))3785      return;37863787    /* Since this is an updated time, we must remove data from3788     * timeouts and then add it again. */3789    Curl_timeouts_remove(&multi->timeouts, data);3790  }37913792  /* Insert the new timer expiry since it is our local minimum. */3793  Curl_timeouts_add(&multi->timeouts, data,3794                    timeouts->offset_us[timeouts->first]);
Curl_expire_set

最近项没有变化时保留原树位置,否则移除后重新登记局部最小期限。

lib/splay.cc00a3021e4b833afebed33f31a8f006a1b7511fe · L190–L200
查看原文件
190  if(root) {191    root = splay(key, root);192    DEBUGASSERT(root);193    if(key == root->key) {194      /* There already exists a node in the tree with the same key.195         Append the new node to the `same` list. */196      struct Curl_tree **panchor = &root->same;197      while(*panchor)198        panchor = &(*panchor)->same;199      *panchor = node;200      return root; /* the root node always stays the same */
splayinsert

相同期限沿 same 链查找尾部再追加;这部分不是常数操作,会影响完整插入的复杂度。

lib/splay.cc00a3021e4b833afebed33f31a8f006a1b7511fe · L222–L230
查看原文件
222void Curl_timeouts_add(struct Curl_timeouts *timeouts,223                       struct Curl_easy *data,224                       timediff_t offset_us)225{226  struct Curl_tree *node = &data->state.timeouts.splaynode;227  DEBUGASSERT(!node->registered);228  timeouts->tree = splayinsert(offset_us, timeouts->tree,229                               node, data->mid);230}
Curl_timeouts_add

全局树节点嵌在 easy handle 中,每个 handle 只登记一个节点。

lib/splay.cc00a3021e4b833afebed33f31a8f006a1b7511fe · L112–L132
查看原文件
112  for(;;) {113    if(key < root->key) {114      /* key is somewhere in root->smaller branch */115      if(!root->smaller)  /* which is empty, done */116        break;117      if(key < root->smaller->key) {118        /* key is somewhere in root->smaller->smaller, make a "Zig step" */119        y = root->smaller;120        root->smaller = y->larger;121        y->larger = root;122        root = y;123        if(!root->smaller)124          break;125      }126      /* Making root->smaller the new root, the old root is no longer127       * referenced. Remember it in the N tree's `r`ight/larger side.128       * Everything in old root is smaller than what the right side129       * of N already has, so it gets added to r->smaller. */130      r->smaller = root;131      r = root;132      root = root->smaller;
splay

查询过程中沿较小分支旋转并重组;较大分支和最终拼接在同函数后文。

lib/splay.cc00a3021e4b833afebed33f31a8f006a1b7511fe · L64–L85
查看原文件
64int Curl_timeouts_next_ms(struct Curl_timeouts *timeouts,65                          const struct curltime *pnow,66                          timediff_t *pexpire_offset_us,67                          uint32_t *pmid)68{69  if(timeouts->tree) { /* splay the lowest key to the root */70    timeouts->tree = splay(TIMEDIFF_T_MIN, timeouts->tree);71  }7273  if(timeouts->tree) {74    timediff_t elapsed_us = Curl_timeouts_offset_us(timeouts, pnow);75    timediff_t delta_us = timeouts->tree->key - elapsed_us;76    if(pmid)77      *pmid = timeouts->tree->id;78    if(pexpire_offset_us)79      *pexpire_offset_us = timeouts->tree->key;80    if(delta_us > 0) { /* expires in the future */81      timediff_t ms = curlx_us_to_ceil_ms(delta_us);82      return (ms > INT_MAX) ? INT_MAX : (int)ms;83    }84    else /* has expired */85      return 0;
Curl_timeouts_next_ms

把最小键移到根后计算剩余时间,微秒向上换算为毫秒。

lib/splay.cc00a3021e4b833afebed33f31a8f006a1b7511fe · L252–L271
查看原文件
252  /* find smallest */253  root = splay(TIMEDIFF_T_MIN, root);254  DEBUGASSERT(root);255  if(key < root->key) {256    /* even the smallest is too big */257    *removed = NULL;258    return root;259  }260261  /* FIRST! Check if there is a list with identical keys */262  if(root->same) {263    x = root->same;264    DEBUGASSERT(x->key == root->key);265    /* 'x' becomes the new root node */266    x->larger = root->larger;267    x->smaller = root->smaller;268    root->same = NULL;269    root->registered = FALSE;270    *removed = root;271    return x; /* new root */
splaygetbest

最小节点仍未到期便停止;相同键链存在时让同期限的下一节点接替根。

lib/multi.cc00a3021e4b833afebed33f31a8f006a1b7511fe · L3145–L3165
查看原文件
3145static CURLMcode add_next_timeout(const struct curltime *pnow,3146                                  struct Curl_multi *multi,3147                                  struct Curl_easy *data)3148{3149  struct expire_timers *timeouts = &data->state.timeouts;3150  timediff_t now_us = Curl_timeouts_offset_us(&multi->timeouts, pnow);31513152  while(timeouts->first < EXPIRE_LAST) {3153    if(timeouts->offset_us[timeouts->first] <= now_us)  /* already expired */3154      timeouts->first = timeouts->next[timeouts->first];3155    else /* timeouts are sorted, first is first in the future now */3156      break;3157  }31583159  if(timeouts->first < EXPIRE_LAST) {3160    /* Insert this node again into the splay. Keep the timer in the list in3161       case we need to recompute future timers. */3162    Curl_timeouts_add(&multi->timeouts, data,3163                      timeouts->offset_us[timeouts->first]);3164  }3165  return CURLM_OK;
add_next_timeout

跳过已经到期的内部定时项,再把该 handle 的下一期限加入树。

lib/multi.cc00a3021e4b833afebed33f31a8f006a1b7511fe · L3179–L3195
查看原文件
3179  while(Curl_timeouts_remove_expired(&multi->timeouts, ts, &mid)) {3180    /* Check if there is one (more) expired timer to deal with! This function3181       extracts a matching node if there is one */3182    data = Curl_multi_get_easy(multi, mid);3183    if(!data) {3184      DEBUGASSERT(0);3185      continue;3186    }3187#ifdef CURLVERBOSE3188    if(CURL_TRC_TIMER_is_verbose(data)) {3189      if(data->state.timeouts.first < EXPIRE_LAST) {3190        CURL_TRC_TIMER(data, data->state.timeouts.first, "has expired");3191      }3192    }3193#endif3194    (void)add_next_timeout(ts, multi, data);3195    Curl_multi_mark_dirty(data);
multi_mark_expired_as_dirty

从全局树提取到期 handle,更新其下一期限并标记状态需要推进。

摘录分别保留调用入口、核心机制及边界;完整上下文见各段源码链接。

收益与取舍#

工程分析

带来的收益

  • 全局索引规模取决于有定时需求的 handle 数,而非所有内部定时项之和
  • 伸展树无需额外平衡信息,访问与旋转合并完成树的调整

付出的代价

  • 不同键的伸展部分为摊还 O(log d),其中 d 是不同期限数;相同期限的插入或指定节点删除还要扫描 same 链,k 个同期限项可带来 O(k) 额外成本,完整操作不能统一写成摊还对数级
  • 到期只是让传输状态机再次运行,并不自动表示请求失败或立即中止连接
  • 同期限大量集中到来时,连续的链尾插入可能积累平方级工作;内部期限维护和回调通知也有独立成本

可比较的选择

带位置索引的最小堆同样适合取最近期限,修改或移除特定 handle 时需要维护它的堆位置。

逐次扫描所有传输实现直接,但每次求最早期限都需要遍历全部活动项。

版本与适用范围#

固定到提交 c00a3021e4b8。分析该提交的 Curl_timeouts_* 封装及 multi.c;此版定时项按 ID 数组和 next 索引组织,不照搬旧版链表描述。仅覆盖超时索引与状态推进,不把超时当作无条件取消。

参考来源#

源码、官方文档与原始论文
01
curl · Curl_expire_set

lib/multi.c · Curl_expire_set

源码c00a3021e4b833afebed33f31a8f006a1b7511fe
02
curl · splayinsert

lib/splay.c · splayinsert

源码c00a3021e4b833afebed33f31a8f006a1b7511fe
03官方文档官方文档;实现以固定提交为准

相关实现与比较#

4 个相关案例
19 个匹配案例