目录
1 背景知识
2 libcurl 基础知识
3 libcurl两种模式
4 libcurl实例分析
正文
1 背景知识:
1.1 基本网络通信cs模式,select 框架,网上例子很多.下面只介绍epoll的难点.其他内容请自行搜索.
1.2 epoll 用法
1.2.1 基础知识:
epoll in/out 与socket io 缓冲区关系？
socket 可读可写是指io 缓冲区的情况,这层由内核控制.socket io 对应的epoll in/out 是应用层.
epoll 通常采用ET模型.ET触发方式是指当fd到状态发生变化时通知,比如:read buffer从无到有,write buffer从满到不满才会通知.
所以用while 不停的读,read完缓冲区.下次来数据的时候,缓冲区又是由无到有,又会触发epoll in.
while 不停的写直到返回缓冲区满返回eagain,然后os立刻会发送,“发送缓冲区的”数据.
“发送缓冲区的”由满变成不满,再次触发epoll out.进入epoll out分支,开始下一轮while写.
如果没写满,但是也写完了,不会再次进入epoll out分支.epoll out的使用时机?
在自己端准备write之前,通过epoll ctrl设置成epoll out.
epoll in 是被动监听接收,epoll out是主动设置.
1.2.2 epoll 模型：网上例子很多.
2 libcurl 基础知识
- libcurl作用:它只能做client. 用于文件上传下载和发送http get,post命令.libcurl不是一个简单的api,是一组api实现的模块,有自己的使用steps. 默认curl是get url 网页,与callback write function连动，写入文件。
- 两种方案:
curl_multi_socket_action()：通常和select/poll/epoll/libev 连用.
curl_multi_perform() + curl_multi_wait().
这个是最简单异步，先发送--等待--接收，这种用法很少用了
https://blog.csdn.net/lijinqi1987/article/details/53925835 and https://blog.csdn.net/Rong_Toa/article/details/105712677.
本文讨论curl_multi_socket_action 方案.
3 libcurl两种模式
**3.1 easy 模式(这种模式比较简单) **
调用curl_easy_setopt函数设置传输的一些基本参数,CULROPT_URL必填.设置完成后,调用curl_easy_perform函数发送数据.
CURLcode curl_easy_setopt(CURL *handle, CURLOPT_WRITEDATA, void *pointer);
curl_easy_setopt(conn->easy, CURLOPT_URL, conn->url);
curl_easy_setopt(conn->easy, CURLOPT_WRITEFUNCTION, write_cb);
curl_easy_setopt(conn->easy, CURLOPT_WRITEDATA, conn);
第一句 curl建立与url 连接，第二句将respond 发送到write_cb,第三句是将write_cb处理内容写入conn指向的文件
3.2 multi 模式
1 ) 常用函数介绍(注意参数)
a ) CURLMOPT_SOCKETFUNCTION:设置socket的回调函数.是所有socket 变化都会调用callback(不仅仅是socket connect，read/write也调用).
int socket_callback(CURL *easy, /* easy handle */
curl_socket_t s, /* socket */
int what, /* describes the socket */
void *userp, /* private callback pointer */
void *socketp); /* private socket pointer */
CURLMOPT_SOCKETDATA: it is params userp
对于接收的话，CURLMOPT_SOCKETDATA没用。socketp是返回值。
CURLMOPT_TIMERDATA and CURLOPT_WRITEDATA 都是对应 curl function的入参.
b) CURLMOPT_TIMERFUNCTION :set callback to receive timeout values
int timer_callback(CURLM *multi, /* multi handle */
long timeout_ms, /* timeout in number of ms */
void *userp); /* private callback pointer */
You can also use the curl_multi_timeout function to poll the value at any given time,
but for an event-based system using the callback is far better than relying on polling the timeout value.
系统超时操作触发callback.
c ) CURLOPT_WRITEFUNCTION:set callback for writing received data
size_t write_callback(char *ptr, size_t size, size_t nmemb, void *userdata);
CURLOPT_WRITEDATA:custom pointer passed to the write callback
设置CURLOPT_WRITEFUNCTION的参数4,把参数1 ptr指向的数据拷到参数4 userdata
d ) curl_multi_setopt
CURLMcode curl_multi_setopt(CURLM *handle, CURLMOPT_TIMERFUNCTION, timer_callback);
CURLMOPT_TIMERDATA:The userp pointer is set with CURLMOPT_TIMERDATA. 入参
curl_multi_setopt is used to tell a libcurl multi handle how to behave.
CURLMOPT_SOCKETFUNCTION,CURLMOPT_SOCKETDATA,CURLMOPT_TIMERFUNCTION,CURLMOPT_TIMERDATA
e) curl_multi_assign
curl_multi_assign(CURLM *multi_handle, curl_socket_t sockfd, void *sockptr);
set association sockfd with sockptr ;curl系统中将sockfd和sockpt结构体绑定
- curl_multi_info_read
Use curl_multi_info_read(3) to figure out which easy handle that completed. 当前只发现这一个功能
注意这个函数不是read data，而是read info ,具体说就是completed flag，是结束检测函数. - 重点:
CURLMcode curl_multi_socket_action(CURLM * multi_handle,
curl_socket_t sockfd,
int ev_bitmask,
int *running_handles);
执行curl命令.curl_multi_socket_action=curl_multi_perform：reads/writes available data given an action.通知libcurl读写数据
执行curl_multi_add_handle中的东东. 注意参数3ev_bitmask是action.
比如: 设置了CURLMOPT_SOCKETFUNCTION就从server download file.设置CURLMOPT_UPLOAD就是上传file.
CURLOPT_WRITEFUNCTION 就调用CURLOPT_WRITEDATA将download data写入file.
4 curl实例:
curl同时和socket，epoll，file 同时打交道.
curl精华：curl_multi_socket_action可以自动get file from server(CURLOPT_URL) and 自动write file to local(CURLOPT_WRITEFUNCTION)
4.1 curl+ epoll
CURLOPT_URL=connect,CURLMOPT_SOCKETFUNCTION=epoll_ctrl, curl_multi_socket_action=get remote file and write file to local ,epoll=select.
实现步骤和时序:
- 创建curl multi handle
- 当timeout的时候,初始化socket
- socket变化触发socekt 回调函数, 设置epoll_ctrl 并且将socket和curl绑定.
- epoll_wait调用epoll_out 分支,curl_multi_socket_action执行发送.
- epoll_in 调用epoll_in 分支,curl_multi_socket_action执行接收.
- curl_multi_info_read 判断是否读完. 读完,删除curl multi handle.
代码:
epoll_ctl(curl_epoll_fd, EPOLL_CTL_ADD, curl_timer_fd, &ev);//put timer into epoll
curl_easy_setopt(curl, CURLOPT_URL, request_uri);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, curl_read_http_body);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, relay_session);
CURLMcode rc = curl_multi_add_handle(curl_multi, curl);
curl_multi_setopt(curl_multi, CURLMOPT_TIMERFUNCTION, curl_timer_setup);//call back for timer
curl_multi_setopt(curl_multi, CURLMOPT_SOCKETFUNCTION, curl_sock_setup);//socket fd 有变化,会调用curl_sock_setup
curl_multi_thread = g_thread_try_new("curl multi thread", &curl_multi_thread_run, NULL, NULL);
static void* curl_multi_thread_run(void* user_data) {
int i, nfds;
struct epoll_event *ev = (struct epoll_event *)malloc(sizeof(struct epoll_event) * MAX_EPOLL_FDS_NUM);
gint64 start_time, end_time;
while (!janus_is_stopping()) {
nfds = epoll_wait(curl_epoll_fd, ev, MAX_EPOLL_FDS_NUM, 3000);//wait 3 seconds
if (nfds > 0) {
for (i = 0; i < nfds; i++) {
curl_event(ev[i].data.fd, ev[i].events);//curl_event//curl_event
}
}
}
}
static void curl_event(int fd, int revents) {
CURLMcode rc;
int still_running = -1;
int action = ((revents & EPOLLIN) ? CURL_CSELECT_IN : 0) |
((revents & EPOLLOUT) ? CURL_CSELECT_OUT : 0);
//call CURLOPT_WRITEFUNCTION callback;read/write curl data
rc = curl_multi_socket_action(curl_multi, fd, action, &still_running);
if (rc == CURLM_OK) {
check_multi_info();//check result
}
}
static void check_multi_info(void) {
CURLMsg *msg = NULL;
int msgs_left;
CURL *curl = NULL;
CURLcode res;
janus_session* session = NULL;
json_error_t error;
json_t *root = NULL;
//从curl msg queue中读取
while ((msg = curl_multi_info_read(curl_multi, &msgs_left))) {
curl = msg->easy_handle;
res = msg->data.result;
curl_easy_getinfo(curl, CURLINFO_PRIVATE, &session);//it is from CURLOPT_PRIVATE
curl_multi_remove_handle(curl_multi, curl);
curl_easy_cleanup(curl);
}
}
https://curl.haxx.se/libcurl/c/ephiperfifo.html
[https://www.jianshu.com/p/80274bc54aff]
(https://www.jianshu.com/p/80274bc54aff)
https://blog.csdn.net/Rong_Toa/article/details/105712677
https://blog.csdn.net/lijinqi1987/article/details/53996129
https://www.cnblogs.com/zl-graduate/articles/6724446.html
