How to apply a standard scaler on your own dataset?

This is only a very short post that contains some tips you need when scaling your data and (maybe) some problems you'll meet during this process.

There are many state-of-the-art libraries can handle this problem easily for you. I'll introduce the one I am mostly familiar with, scikit-learn in Python.

This is the most top 5 rows in our sample dataset, where open, high, 'low', 'volume' and 'amount' are our features and close is the target we want to be able to predict after the model is trained.

Data Example

But wait, before we start throwing our data into the model training process, what did you forget?

You need to standardize features by removing the mean and scaling to unit variance.

def standard_scaler(X_train, X_test):
    train_samples, train_nx, train_ny = X_train.shape
    test_samples, test_nx, test_ny = X_test.shape
    X_train = X_train.reshape((train_samples, train_nx * train_ny))
    X_test = X_test.reshape((test_samples, test_nx * test_ny))
    preprocessor = prep.StandardScaler().fit(X_train)
    X_train = preprocessor.transform(X_train)
    X_test = preprocessor.transform(X_test)
    X_train = X_train.reshape((train_samples, train_nx, train_ny))
    X_test = X_test.reshape((test_samples, test_nx, test_ny))
    return X_train, X_test

TODO...
TODO...
TODO...

最后编辑于
©著作权归作者所有,转载或内容合作请联系作者
平台声明:文章内容(如有图片或视频亦包括在内)由作者上传并发布,文章内容仅代表作者本人观点,简书系信息发布平台,仅提供信息存储服务。

推荐阅读更多精彩内容

  • **2014真题Directions:Read the following text. Choose the be...
    又是夜半惊坐起阅读 9,900评论 0 23
  • 我是一尾失了光明的游鱼 游弋在干涸裂开的尘世里 光与影交替而行 风劈开爱与迷离 唯你与信仰历久弥新 我抛弃海洋寻找...
    荆予阅读 257评论 0 0
  • 第二天早晨,先去吃了早饭,然后到外面找车子,想去塔克拉玛干沙漠看看。先是问了一些司机,David 和Gold...
    绿洲121212阅读 692评论 0 3
  • 一个梦在春天里 睁开惺忪的睡眼 在温暖的阳光里舒展着那踡缩的身体 在蜜蜂花朵鱼儿小溪编织的的童话里 陶醉着和煦的春...
    慕容兰馨阅读 225评论 5 18
  • (1)面部护理 1)保湿润肤 3D天竺葵+2D甜橙+10ML甜杏仁油 2)防敏抗敏 3D洋甘菊+2D茉莉+10ML...
    xmxin_阅读 650评论 0 1