数据量10000行以下
-
DataStudio页面下载(对应工作空间需开启下载设置)
import java.util.List;
import com.aliyun.odps.Instance;
import com.aliyun.odps.Odps;
import com.aliyun.odps.OdpsException;
import com.aliyun.odps.account.Account;
import com.aliyun.odps.account.AliyunAccount;
import com.aliyun.odps.data.Record;
import com.aliyun.odps.task.SQLTask;
public class testSql {
private static final String accessId = "";
private static final String accessKey = "";
private static final String endPoint = "http://service.odps.aliyun.com/api";
private static final String project = "";
private static final String sql = "select category from iris;";
public static void
main(String[] args) {
Account account = new AliyunAccount(accessId, accessKey);
Odps odps = new Odps(account);
odps.setEndpoint(endPoint);
odps.setDefaultProject(project);
Instance i;
try {
i = SQLTask.run(odps, sql);
i.waitForSuccess();
List<Record> records = SQLTask.getResult(i);
for(Record r:records){
System.out.println(r.get(0).toString());
}
} catch (OdpsException e) {
e.printStackTrace();
}
}
}
数据量10000行以上
- 本地spark读取odps表，写本地文件
- 离线数据集成到OSS, 然后通过OSS下载到本地
- 使用JDBC方式连接MaxCompute
- 使用批量数据通道SDK
<dependency>
<groupId>com.aliyun.odps</groupId>
<artifactId>odps-sdk-core</artifactId>
<version>0.31.3-public</version>
</dependency>
注意点：是用0.24.0版本只支持一下数据类型 BIGINT, DOUBLE, BOOLEAN, DATETIME, STRING, DECIMAL, MAP, ARRAY;
报错如下：
ErrorCode=Local Error, ErrorMessage=Invalid json content.
at com.aliyun.odps.tunnel.TableTunnel$DownloadSession.loadFromJson(TableTunnel.java:1483)
at com.aliyun.odps.tunnel.TableTunnel$DownloadSession.initiate(TableTunnel.java:1334)
at com.aliyun.odps.tunnel.TableTunnel$DownloadSession.<init>(TableTunnel.java:1215)
at com.aliyun.odps.tunnel.TableTunnel.createDownloadSession(TableTunnel.java:371)
at TunnelCheck.main(TunnelCheck.java:36)
Caused by: java.lang.IllegalArgumentException: No enum constant com.aliyun.odps.OdpsType.INT
at java.lang.Enum.valueOf(Enum.java:238)
at com.aliyun.odps.OdpsType.valueOf(OdpsType.java:27)
at com.aliyun.odps.tunnel.TunnelTableSchema.parseColumn(TunnelTableSchema.java:60)
at com.aliyun.odps.tunnel.TunnelTableSchema.<init>(TunnelTableSchema.java:36)
at com.aliyun.odps.tunnel.TableTunnel$DownloadSession.loadFromJson(TableTunnel.java:1480)
... 4 more
