Advertisement
Previous work has cast doubt on the general framework of uniform convergence and its ability to explain generalization in neural networks. By considering a specific dataset, it was observed that a neural network completely misclassifies a projection of the training data (adversarial set), rendering any existing generalization bound based on uniform convergence vacuous. We provide an extensive theoretical investigation of the previously studied data setting through the lens of infinitely-wide models. We prove that the Neural Tangent Kernel (NTK) also suffers from the same phenomenon and we uncover its origin. We highlight the important role of the output bias and show theoretically as well as empirically how a sensible choice completely mitigates the problem. We identify sharp phase transitions in the accuracy on the adversarial set and study its dependency on the training sample size. As a result, we are able to characterize critical sample sizes beyond which the effect disappears. Moreover, we study decompos🌟 Prepare for an extraordinary experience at Carol's Pub for Simple Remedy, a mesmerizing event event that promises to redefine your expectations. From captivating presentations to interactive showcases, this event in Illinois, United States is designed to leave you inspired and amazed. Join us for this incredible event 🚀. http://maps.seatics.com/CarolsPub_ResTable-ResBooth-GA_2023-02-28_2023-02-28_0831_02282023_084602_SVGC_tn.png
Advertisement
Event Venue & Nearby Stays
Carol's Pub, 4959 North Clark Street, Carol's Pub, Chicago, Illinois, United States
Tickets
