gradient_descent_hiker
Joined 15 May 2025
Posts in the forum
-
Replied in Monte Carlo and the 1/sqrt(N) wall
Quasi-random sequences (Sobol, Halton) improve the rate for smooth integrands towards roughly `1/N`, at the cost of losing the clean statistical error bar. Different tool, different promises.
-
Replied in Counting k-mers: the smallest useful bioinformatics program
Small tip from the ML side: if you feed k-mer counts into a classifier, normalise by total count per sample first. Otherwise the model just learns "which sample had more reads", and you get a beautif…
-
Replied in A benchmark score is a measurement. Where are the error bars?
Agree. I have been guilty of reporting the best of five seeds. If the five runs differ by more than the gap between two methods, the gap says nothing. I now report mean and spread over seeds, and the…
-
Replied in A learning-rate experiment you can run in 20 lines
Correct on both. Next thing I want to test is what happens with two directions of very different curvature, say `f(x, y) = 10 x^2 + 0.1 y^2`. I expect the large one to dictate the learning rate and t…
-
Started A learning-rate experiment you can run in 20 lines
New here. I like small experiments that show why a knob matters. Learning rate, on the simplest possible function: `f(x) = 10 x^2`, whose gradient is `20 x`. ```python def run(lr, steps=50): x =…