modified‎ ‎step‎ ‎size‎ ‎for‎ ‎enhanced‎ ‎stochastic gradient descent‎: ‎convergence and experiments

‎this paper introduces a novel approach to enhance the performance of the stochastic gradient descent (sgd) algorithm by incorporating a modified decay step size based on frac{1}{sqrt{t}}‎. ‎the proposed step size integrates a logarithmic term‎, ‎leading to the selection of smaller values in the final iterations‎. ‎our analysis establishes a convergence rate of o(frac{ln t}{sqrt{t}}) for smooth non-convex functions without the polyak-łojasiewicz condition‎. ‎to evaluate the effectiveness of our approach‎, ‎we conducted numerical experiments on image classification tasks using the fashion-mnist and cifar10 datasets‎, ‎and the results demonstrate significant improvements in accuracy‎, ‎with enhancements of $0.5% and 1.4% observed‎, ‎respectively‎, ‎compared to the traditional frac{1}{sqrt{t}} step size‎. ‎the source code can be found at اttps://github.com/shamaeem/lnsqrtstepsize.