Machine Learning

from sklearn.preprocessing import StandardScaler| https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html | fit the scaler on the TRAINING split only, then .transform() train/val/test with it -- fitting on all the data leaks test-set statistics into training |'ml_data1'

train_df, val_df, test_df = df.iloc[:n_train], df.iloc[n_train:n_train+n_val], df.iloc[n_train+n_val:]| | for time series, split chronologically (never shuffle before splitting) -- the test set must be strictly after training in time, or the model is evaluated on data it indirectly saw |'ml_data2'

fut = df.groupby(day_key)['return'].shift(-horizon)| | when a label looks ahead N steps, group by any natural boundary first (day, session, entity id) so the shift never reads across it -- otherwise the last row(s) of each group get labeled from data outside their own context |'ml_leak1'

from sklearn.metrics import classification_report| https://scikit-learn.org/stable/modules/generated/sklearn.metrics.classification_report.html | always compare accuracy against the majority-class baseline (support of the biggest class / total) -- a model can beat 50% accuracy while doing worse than "always predict the majority class" |'ml_leak2'