presto(一) -- 启动Server流程

Server(presto-0.207)启动涉及到的代码不是很多(这里只讨论大致的执行流程,不涉及具体的实现细节),主要是关于Plugins的加载,读配置文件.properties,进行相应的操作。
首先,根据presto日志server.log定位服务启动的入口。
INFO main com.facebook.presto.server.PrestoServer ======== SERVER STARTED ========
启动类:presto-main/src/main/java/com/facebook/presto/server/PrestoServer.java
server启动的所有流程均在run()中实现:
首先,虚拟机参数相关验证,比如:是否JDK8+、JDK厂商:Oracle or OpenJDK、64-bit JVM版本、x86_64 on Mac OS,或者x86-64、amd64 on Linux系统,是否采用G1的垃圾回收器等。系统时间要求:必须是当前时间。

verifyJvmRequirements();
verifySystemTimeIsReasonable();

接着,创建不可变集合ImmutableList,传入一些启动模块,具体的实现细节,后续有需要再分析。

ImmutableList.Builder<Module> modules = ImmutableList.builder();
        modules.add(
                new NodeModule(),
                new DiscoveryModule(),
                new HttpServerModule(),
                new JsonModule(),
                new JaxrsModule(true),
                new MBeanModule(),
                new JmxModule(),
                new JmxHttpModule(),
                new LogJmxModule(),
                new TraceTokenModule(),
                new JsonEventModule(),
                new HttpEventModule(),
                new ServerSecurityModule(),
                new AccessControlModule(),
                new EventListenerModule(),
                new ServerMainModule(sqlParserOptions),
                new GracefulShutdownModule());

        modules.addAll(getAdditionalModules());
        //使用io.airlift.平台代码库构建的应用程序的入口点,并加载需要的module实例(包含一些启动默认参数)
        Bootstrap app = new Bootstrap(modules.build());

关于启动的默认参数比较多,现截取一部分:

2018-08-24T10:54:27.769+0800    INFO    main    Bootstrap   PROPERTY                                                              DEFAULT                           RUNTIME                                  DESCRIPTION
2018-08-24T10:54:27.769+0800    INFO    main    Bootstrap   event.max-output-stage-size                                           16MB                              16MB
2018-08-24T10:54:27.769+0800    INFO    main    Bootstrap   query.client.timeout                                                  5.00m                             5.00m
2018-08-24T10:54:27.769+0800    INFO    main    Bootstrap   query.initial-hash-partitions                                         100                               100
2018-08-24T10:54:27.769+0800    INFO    main    Bootstrap   query-manager.initialization-required-workers                         1                                 1                                        Minimum number of workers that must be available before the cluster will accept queries
2018-08-24T10:54:27.769+0800    INFO    main    Bootstrap   query-manager.initialization-timeout                                  5.00m                             5.00m                                    After this time, the cluster will accept queries even if the minimum required workers are not available
2018-08-24T10:54:27.769+0800    INFO    main    Bootstrap   query.max-concurrent-queries                                          1000                              1000
2018-08-24T10:54:27.769+0800    INFO    main    Bootstrap   query.max-history                                                     100                               100
2018-08-24T10:54:27.769+0800    INFO    main    Bootstrap   query.max-length                                                      1000000                           1000000
2018-08-24T10:54:27.769+0800    INFO    main    Bootstrap   query.max-queued-queries                                              5000                              5000
2018-08-24T10:54:27.769+0800    INFO    main    Bootstrap   query.min-expire-age                                                  15.00m                            15.00m
2018-08-24T10:54:27.769+0800    INFO    main    Bootstrap   query.min-schedule-split-batch-size                                   100                               100
2018-08-24T10:54:27.769+0800    INFO    main    Bootstrap   query.execution-policy                                                all-at-once                       all-at-once
2018-08-24T10:54:27.770+0800    INFO    main    Bootstrap   query.manager-executor-pool-size                                      5                                 5
2018-08-24T10:54:27.770+0800    INFO    main    Bootstrap   query.max-cpu-time                                                    1000000000.00d                    1000000000.00d
2018-08-24T10:54:27.770+0800    INFO    main    Bootstrap   query.max-execution-time                                              100.00d                           100.00d
2018-08-24T10:54:27.770+0800    INFO    main    Bootstrap   query.max-run-time                                                    100.00d                           100.00d
2018-08-24T10:54:27.770+0800    INFO    main    Bootstrap   query.remote-task.max-callback-threads                                1000                              1000
2018-08-24T10:54:27.770+0800    INFO    main    Bootstrap   query.remote-task.max-error-duration                                  5.00m                             5.00m
2018-08-24T10:54:27.770+0800    INFO    main    Bootstrap   query.remote-task.min-error-duration                                  5.00m                             5.00m
2018-08-24T10:54:27.770+0800    INFO    main    Bootstrap   query.schedule-split-batch-size                                       1000                              1000
2018-08-24T10:54:27.770+0800    INFO    main    Bootstrap   task.client.timeout                                                   2.00m                             2.00m
2018-08-24T10:54:27.770+0800    INFO    main    Bootstrap   task.http-response-threads                                            100                               100
2018-08-24T10:54:27.770+0800    INFO    main    Bootstrap   task.http-timeout-threads                                             3                                 3
2018-08-24T10:54:27.770+0800    INFO    main    Bootstrap   task.info.max-age                                                     15.00m                            15.00m
2018-08-24T10:54:27.770+0800    INFO    main    Bootstrap   task.info-update-interval                                             3.00s                             3.00s                                    Interval between updating task data
2018-08-24T10:54:27.770+0800    INFO    main    Bootstrap   task.initial-splits-per-node                                          64                                6
2018-08-24T10:54:27.770+0800    INFO    main    Bootstrap   task.level-time-multiplier                                            2                                 2                                        Factor that determines the target scheduled time for a level relative to the next
2018-08-24T10:54:27.770+0800    INFO    main    Bootstrap   task.max-drivers-per-task                                             2147483647                        2147483647                               Maximum number of drivers a task can run
2018-08-24T10:54:27.770+0800    INFO    main    Bootstrap   task.max-index-memory                                                 64MB                              64MB
2018-08-24T10:54:27.771+0800    INFO    main    Bootstrap   task.max-local-exchange-buffer-size                                   32MB                              32MB
2018-08-24T10:54:27.771+0800    INFO    main    Bootstrap   driver.max-page-partitioning-buffer-size                              32MB                              32MB
2018-08-24T10:54:27.771+0800    INFO    main    Bootstrap   task.max-partial-aggregation-memory                                   16MB                              16MB
2018-08-24T10:54:27.771+0800    INFO    main    Bootstrap   task.max-worker-threads                                               64                                6
2018-08-24T10:54:27.771+0800    INFO    main    Bootstrap   task.min-drivers                                                      128                               12
2018-08-24T10:54:27.771+0800    INFO    main    Bootstrap   task.min-drivers-per-task                                             3                                 3                                        Minimum number of drivers guaranteed to run per task given there is sufficient work to do
2018-08-24T10:54:27.771+0800    INFO    main    Bootstrap   task.share-index-loading                                              false                             false
2018-08-24T10:54:27.771+0800    INFO    main    Bootstrap   sink.max-buffer-size                                                  32MB                              1GB
2018-08-24T10:54:27.771+0800    INFO    main    Bootstrap   task.split-concurrency-adjustment-interval                            100.00ms                          100.00ms
2018-08-24T10:54:27.771+0800    INFO    main    Bootstrap   task.status-refresh-max-wait                                          1.00s                             1.00s
2018-08-24T10:54:27.771+0800    INFO    main    Bootstrap   task.concurrency                                                      16                                16                                       Default number of local parallel jobs per worker
2018-08-24T10:54:27.771+0800    INFO    main    Bootstrap   task.cpu-timer-enabled                                                true                              true
2018-08-24T10:54:27.771+0800    INFO    main    Bootstrap   task.task-notification-threads                                        5                                 5                                        Number of threads used for internal task event notifications
2018-08-24T10:54:27.771+0800    INFO    main    Bootstrap   task.task-yield-threads                                               3                                 3                                        Number of threads used for setting yield signals
2018-08-24T10:54:27.772+0800    INFO    main    Bootstrap   task.verbose-stats                                                    false                             false

开始加载plugins和配置文件

try {
            Injector injector = app.strictConfig().initialize();
            //生成PluginManager实例,载入/data0/presto/data/plugin目录下的.jar
            injector.getInstance(PluginManager.class).loadPlugins();
            //生成StaticCatalogStore实例,载入.properties配置文件
            injector.getInstance(StaticCatalogStore.class).loadCatalogs();

            // TODO: remove this huge hack
            //更新UUID, ServiceAnnouncement
            updateConnectorIds(
                    injector.getInstance(Announcer.class),
                    injector.getInstance(CatalogManager.class),
                    injector.getInstance(ServerConfig.class),
                    injector.getInstance(NodeSchedulerConfig.class));
            //接口,无具体方法实现
            injector.getInstance(SessionSupplier.class).loadConfigurationManager();
            injector.getInstance(ResourceGroupManager.class).loadConfigurationManager();
            //生成AccessControlManager实例,如果路径etc/下存在文件access-control.properties
            //如果检查通过,或者不存在该文件,打印日志信息"-- Loaded system access control allow-all --"
            injector.getInstance(AccessControlManager.class).loadSystemAccessControl();
            //原理过程大致同上
            injector.getInstance(PasswordAuthenticatorManager.class).loadPasswordAuthenticator();
            injector.getInstance(EventListenerManager.class).loadConfiguredEventListener();

            injector.getInstance(Announcer.class).start();

            log.info("======== SERVER STARTED ========");
        }
        catch (Throwable e) {
            log.error(e);
            System.exit(1);
        }
    }

关于AccessControlManager.class下的loadSystemAccessControl()代码分析:

public void loadSystemAccessControl()
            throws Exception
    {
        if (ACCESS_CONTROL_CONFIGURATION.exists()) {
            Map<String, String> properties = new HashMap<>(loadProperties(ACCESS_CONTROL_CONFIGURATION));

            String accessControlName = properties.remove(ACCESS_CONTROL_PROPERTY_NAME);
            checkArgument(!isNullOrEmpty(accessControlName),
                    "Access control configuration %s does not contain %s", ACCESS_CONTROL_CONFIGURATION.getAbsoluteFile(), ACCESS_CONTROL_PROPERTY_NAME);

            setSystemAccessControl(accessControlName, properties);
        }
        else {
            setSystemAccessControl(AllowAllSystemAccessControl.NAME, ImmutableMap.of());
        }
    }

    @VisibleForTesting
    protected void setSystemAccessControl(String name, Map<String, String> properties)
    {
        requireNonNull(name, "name is null");
        requireNonNull(properties, "properties is null");

        checkState(systemAccessControlLoading.compareAndSet(false, true), "System access control already initialized");

        log.info("-- Loading system access control --");

        SystemAccessControlFactory systemAccessControlFactory = systemAccessControlFactories.get(name);
        checkState(systemAccessControlFactory != null, "Access control %s is not registered", name);

        SystemAccessControl systemAccessControl = systemAccessControlFactory.create(ImmutableMap.copyOf(properties));
        this.systemAccessControl.set(systemAccessControl);

        log.info("-- Loaded system access control %s --", name);
    }

由于在集群中没有配置该文件,启动时默认allow-all,直接通过检查。后面两步的加载原理大致也是一样。运行到这里,就可以启动server了。

injector.getInstance(Announcer.class).start();
log.info("======== SERVER STARTED ========");
最后编辑于
©著作权归作者所有,转载或内容合作请联系作者
  • 序言:七十年代末,一起剥皮案震惊了整个滨河市,随后出现的几起案子,更是在滨河造成了极大的恐慌,老刑警刘岩,带你破解...
    沈念sama阅读 220,367评论 6 512
  • 序言:滨河连续发生了三起死亡事件,死亡现场离奇诡异,居然都是意外死亡,警方通过查阅死者的电脑和手机,发现死者居然都...
    沈念sama阅读 93,959评论 3 396
  • 文/潘晓璐 我一进店门,熙熙楼的掌柜王于贵愁眉苦脸地迎上来,“玉大人,你说我怎么就摊上这事。” “怎么了?”我有些...
    开封第一讲书人阅读 166,750评论 0 357
  • 文/不坏的土叔 我叫张陵,是天一观的道长。 经常有香客问我,道长,这世上最难降的妖魔是什么? 我笑而不...
    开封第一讲书人阅读 59,226评论 1 295
  • 正文 为了忘掉前任,我火速办了婚礼,结果婚礼上,老公的妹妹穿的比我还像新娘。我一直安慰自己,他们只是感情好,可当我...
    茶点故事阅读 68,252评论 6 397
  • 文/花漫 我一把揭开白布。 她就那样静静地躺着,像睡着了一般。 火红的嫁衣衬着肌肤如雪。 梳的纹丝不乱的头发上,一...
    开封第一讲书人阅读 51,975评论 1 308
  • 那天,我揣着相机与录音,去河边找鬼。 笑死,一个胖子当着我的面吹牛,可吹牛的内容都是我干的。 我是一名探鬼主播,决...
    沈念sama阅读 40,592评论 3 420
  • 文/苍兰香墨 我猛地睁开眼,长吁一口气:“原来是场噩梦啊……” “哼!你这毒妇竟也来了?” 一声冷哼从身侧响起,我...
    开封第一讲书人阅读 39,497评论 0 276
  • 序言:老挝万荣一对情侣失踪,失踪者是张志新(化名)和其女友刘颖,没想到半个月后,有当地人在树林里发现了一具尸体,经...
    沈念sama阅读 46,027评论 1 319
  • 正文 独居荒郊野岭守林人离奇死亡,尸身上长有42处带血的脓包…… 初始之章·张勋 以下内容为张勋视角 年9月15日...
    茶点故事阅读 38,147评论 3 340
  • 正文 我和宋清朗相恋三年,在试婚纱的时候发现自己被绿了。 大学时的朋友给我发了我未婚夫和他白月光在一起吃饭的照片。...
    茶点故事阅读 40,274评论 1 352
  • 序言:一个原本活蹦乱跳的男人离奇死亡,死状恐怖,灵堂内的尸体忽然破棺而出,到底是诈尸还是另有隐情,我是刑警宁泽,带...
    沈念sama阅读 35,953评论 5 347
  • 正文 年R本政府宣布,位于F岛的核电站,受9级特大地震影响,放射性物质发生泄漏。R本人自食恶果不足惜,却给世界环境...
    茶点故事阅读 41,623评论 3 331
  • 文/蒙蒙 一、第九天 我趴在偏房一处隐蔽的房顶上张望。 院中可真热闹,春花似锦、人声如沸。这庄子的主人今日做“春日...
    开封第一讲书人阅读 32,143评论 0 23
  • 文/苍兰香墨 我抬头看了看天上的太阳。三九已至,却和暖如春,着一层夹袄步出监牢的瞬间,已是汗流浃背。 一阵脚步声响...
    开封第一讲书人阅读 33,260评论 1 272
  • 我被黑心中介骗来泰国打工, 没想到刚下飞机就差点儿被人妖公主榨干…… 1. 我叫王不留,地道东北人。 一个月前我还...
    沈念sama阅读 48,607评论 3 375
  • 正文 我出身青楼,却偏偏与公主长得像,于是被迫代替她去往敌国和亲。 传闻我的和亲对象是个残疾皇子,可洞房花烛夜当晚...
    茶点故事阅读 45,271评论 2 358

推荐阅读更多精彩内容