python 批量上传文件到阿里云oss，并写入Excel，存到本地

最近公司要往阿里云oss上传视频，大小差不多有200G，原先让运营去一个一个的添加，但是这能麻烦死人，所以就让技术去批量上传。所以研究了一下用python往oss上传视频

首先需引用以下几个模块

pip install oss2
pip install tablib
pip install pyexcel-xlsx

其次因为本地的视频都是比如中文.mp4这样的，所以引入一个随机字符串

import random
def generate_random_str(randomlength=8):
    """
    生成一个指定长度的随机字符串
    """
    random_str = ''
    base_str = 'ABCDEFGHIGKLMNOPQRSTUVWXYZabcdefghigklmnopqrstuvwxyz0123456789'
    length = len(base_str) - 1
    for i in range(randomlength):
        random_str += base_str[random.randint(0, length)]
    return random_str

因为是文件夹里面套文件夹，但是只上传文件，所以需要获取子文件夹下面的视频

代码如下：

def upload(dir):
    fs = os.listdir(dir)
    for f in fs:
        file = dir + "/" + f
        if os.path.isdir(file):
            upload(file)
        else:
            if 'DS_Store' not in file and 'png' not in f and 'JPG' not in f:
                putAliyun(file, f)

备注：因为文件夹里面还有图片，所以去除后缀为png、JPG的图片。因为用的是mac上传，所以文件夹里面有.DS_Store，所以也需要去除

所以这个的整个代码如下:

#!/usr/bin/env python
# ! -*- coding:utf-8 -*-

import oss2
import random
import os
import tablib
import time

ossDir = '/Users/FQY/Desktop/upload'
key = XXX
secret = XXX
bucketname = XXX

dataset = tablib.Dataset()
header = ('title', 'url')
dataset.headers = header

def generate_random_str(randomlength=8):
    """
    生成一个指定长度的随机字符串
    """
    random_str = ''
    base_str = 'ABCDEFGHIGKLMNOPQRSTUVWXYZabcdefghigklmnopqrstuvwxyz0123456789'
    length = len(base_str) - 1
    for i in range(randomlength):
        random_str += base_str[random.randint(0, length)]
    return random_str

def getmkname(path):
    remoteName = path.replace(ossDir, '')
    dir_names = remoteName.split('/')
    dir_names.pop()
    res = filter(None, dir_names)
    mkdir_name = '-'.join(res)
    return mkdir_name

#获得上传的时长
def progress_callback(bytes_consumed, total_bytes):
    print('bytes_consumed is {}'.format(bytes_consumed))
    print('total_bytes is {}'.format(total_bytes))


def putAliyun(path, f):
    key = 'language/' + str(int(time.time())) + generate_random_str() + '.mp4'
    auth = oss2.Auth(key, secret)
    bucket = oss2.Bucket(auth, 'http://oss-cn-hangzhou.aliyuncs.com', bucketname)
    result = bucket.put_object_from_file(key=key, filename=path,progress_callback=progress_callback)

    if result.status == 200:
        aliyun = 'http://video.oss-cn-hangzhou.aliyuncs.com/{}'.format(key)
        title = '【{}】{}'.format(getmkname(path), f.split('.mp4')[0])
        dataset.append([title, aliyun])
    else:
        print('upload fail,error code', result.status)


def upload(dir):
    fs = os.listdir(dir)
    for f in fs:
        file = dir + "/" + f
        if os.path.isdir(file):
            upload(file)
        else:
            if 'DS_Store' not in file and 'png' not in f and 'JPG' not in f:
                putAliyun(file, f)

upload(ossDir)

myfile = open('/Users/FQY/Desktop/mydata_video.xlsx', 'wb')
myfile.write(dataset.xlsx)
myfile.close()

tablib的介绍，可以观看以下的文章
利用tablib、make_response 进行文件的下载

最后编辑于：2019.02.27 16:48:38