{"id":53,"date":"2019-06-13T15:23:57","date_gmt":"2019-06-13T15:23:57","guid":{"rendered":"https:\/\/losslandscape.com\/?page_id=53"},"modified":"2021-03-13T22:55:37","modified_gmt":"2021-03-13T22:55:37","slug":"knowledge","status":"publish","type":"page","link":"https:\/\/losslandscape.com\/knowledge\/","title":{"rendered":"Knowledge"},"content":{"rendered":"<p>[et_pb_section fb_built=&#8221;1&#8243; admin_label=&#8221;Header&#8221; _builder_version=&#8221;3.23.3&#8243; use_background_color_gradient=&#8221;on&#8221; background_color_gradient_start=&#8221;#ffffff&#8221; background_color_gradient_end=&#8221;rgba(255,255,255,0)&#8221; background_color_gradient_direction=&#8221;188deg&#8221; background_color_gradient_end_position=&#8221;45%&#8221; background_color_gradient_overlays_image=&#8221;on&#8221; background_image=&#8221;https:\/\/losslandscape.com\/wp-content\/uploads\/2019\/10\/knowledgeback.jpg&#8221; width=&#8221;100%&#8221; max_width=&#8221;100%&#8221; max_width_last_edited=&#8221;on|desktop&#8221; custom_padding=&#8221;0px||0px||true|false&#8221;][et_pb_row use_custom_gutter=&#8221;on&#8221; custom_padding=&#8221;0px||||false&#8221; make_equal=&#8221;on&#8221; custom_padding_last_edited=&#8221;off|phone&#8221; _builder_version=&#8221;3.22.7&#8243; background_image=&#8221;https:\/\/losslandscape.com\/wp-content\/uploads\/2019\/05\/calligraphy_brushes_17.png&#8221; background_position=&#8221;bottom_center&#8221; width=&#8221;100%&#8221; max_width=&#8221;100%&#8221; min_height=&#8221;27px&#8221;][et_pb_column type=&#8221;4_4&#8243; _builder_version=&#8221;3.22.5&#8243;][et_pb_text _builder_version=&#8221;3.23.3&#8243; text_font=&#8221;Allura||||||||&#8221; text_font_size=&#8221;28vw&#8221; text_line_height=&#8221;0.2em&#8221; header_3_font=&#8221;||||||||&#8221; text_orientation=&#8221;right&#8221; filter_opacity=&#8221;9%&#8221;]<\/p>\n<p>Loss<\/p>\n<p>[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row custom_padding_tablet=&#8221;17vw||17vw||false&#8221; custom_margin=&#8221;4vw|||&#8221; custom_padding_last_edited=&#8221;off|phone&#8221; padding_top_bottom_link_1=&#8221;true&#8221; padding_left_right_link_1=&#8221;true&#8221; padding_1_tablet=&#8221;60px||60px||true&#8221; padding_1_phone=&#8221;40px||40px||true&#8221; padding_1_last_edited=&#8221;off|phone&#8221; _builder_version=&#8221;3.22.7&#8243; background_position=&#8221;bottom_center&#8221; max_width=&#8221;80%&#8221; use_custom_width=&#8221;on&#8221; width_unit=&#8221;off&#8221; custom_width_percent=&#8221;100%&#8221;][et_pb_column type=&#8221;4_4&#8243; _builder_version=&#8221;3.12.2&#8243;][et_pb_text _builder_version=&#8221;3.23.3&#8243; text_font=&#8221;||||||||&#8221; text_text_color=&#8221;#ffffff&#8221; header_font=&#8221;Muli|800||on|||||&#8221; header_text_color=&#8221;rgba(255,255,255,0.56)&#8221; header_font_size=&#8221;22px&#8221; header_letter_spacing=&#8221;0.5em&#8221; header_line_height=&#8221;2em&#8221; max_width=&#8221;300px&#8221; max_width_tablet=&#8221;70%&#8221; max_width_last_edited=&#8221;off|phone&#8221; animation_style=&#8221;zoom&#8221; animation_intensity_zoom=&#8221;5%&#8221; header_font_size_tablet=&#8221;12px&#8221; header_font_size_phone=&#8221;11px&#8221; header_font_size_last_edited=&#8221;on|phone&#8221; header_letter_spacing_last_edited=&#8221;off|desktop&#8221;]<\/p>\n<h1>Knowledge<\/h1>\n<p>[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row custom_padding=&#8221;||0px|||&#8221; custom_margin=&#8221;8vw|||||&#8221; _builder_version=&#8221;3.22.7&#8243; width=&#8221;100%&#8221; max_width=&#8221;100%&#8221;][et_pb_column type=&#8221;4_4&#8243; _builder_version=&#8221;3.22&#8243;][et_pb_image src=&#8221;https:\/\/losslandscape.com\/wp-content\/uploads\/2019\/05\/calligraphy_54.png&#8221; force_fullwidth=&#8221;on&#8221; align_tablet=&#8221;center&#8221; align_last_edited=&#8221;on|desktop&#8221; _builder_version=&#8221;3.23&#8243; custom_padding=&#8221;||0px|||&#8221;][\/et_pb_image][\/et_pb_column][\/et_pb_row][\/et_pb_section][et_pb_section fb_built=&#8221;1&#8243; custom_padding_last_edited=&#8221;on|phone&#8221; admin_label=&#8221;Header&#8221; _builder_version=&#8221;3.22.7&#8243; background_size=&#8221;initial&#8221; background_position=&#8221;top_center&#8221; custom_margin=&#8221;||&#8221; custom_padding=&#8221;4px|||||&#8221; custom_padding_phone=&#8221;30px|||&#8221; locked=&#8221;off&#8221;][et_pb_row custom_padding=&#8221;13px|||||&#8221; _builder_version=&#8221;3.22.3&#8243; locked=&#8221;off&#8221;][et_pb_column type=&#8221;4_4&#8243; _builder_version=&#8221;3.22&#8243;][et_pb_text _builder_version=&#8221;3.23.3&#8243;]<\/p>\n<p>This page contains papers and related content linked to the loss landscape challenge. If you have a paper or article that you would like to share, email ideami@ideami.com.<\/p>\n<p>[\/et_pb_text][et_pb_text _builder_version=&#8221;3.23.3&#8243; text_font=&#8221;Muli||||||||&#8221; text_font_size=&#8221;17px&#8221; text_line_height=&#8221;1.5em&#8221; header_font=&#8221;||||||||&#8221; width_tablet=&#8221;100%&#8221; width_last_edited=&#8221;off|desktop&#8221; module_alignment=&#8221;left&#8221; custom_padding=&#8221;0px||0px|||&#8221; text_font_size_phone=&#8221;15px&#8221; text_font_size_last_edited=&#8221;on|desktop&#8221; text_line_height_phone=&#8221;1.7em&#8221; text_line_height_last_edited=&#8221;on|phone&#8221;]<\/p>\n<ul>\n<li><strong>Why Flatness Correlates With Generalization For Deep Neural Networks<br \/><\/strong>huofeng Zhang, Isaac Reid, Guillermo Valle P\u00e9rez, Ard Louis<strong><br \/><\/strong>Submitted on 10 March, 2021;<strong><br \/><a href=\"https:\/\/arxiv.org\/abs\/2103.06219\" target=\"_blank\" rel=\"noopener noreferrer\">arXiv:2103.06219 <\/a><\/strong><\/li>\n<\/ul>\n<p><strong><\/strong><\/p>\n<ul>\n<li><strong>Quantum Earth Mover&#8217;s Distance: A New Approach to Learning Quantum Data<br \/><\/strong>Bobak Toussi Kiani, Giacomo De Palma, Milad Marvian, Zi-Wen Liu, Seth Lloyd<strong><br \/><\/strong>Submitted on 08 Jan, 2021;<strong><br \/><a href=\"https:\/\/arxiv.org\/abs\/2101.03037\" target=\"_blank\" rel=\"noopener noreferrer\">arXiv:\u00a02101.03037<\/a><\/strong><\/li>\n<\/ul>\n<p><strong><\/strong><\/p>\n<ul>\n<li><strong>Towards a Better Global Loss Landscape of GANs<br \/><\/strong>Ruoyu Sun, Tiantian Fang, Alex Schwing<strong><br \/><\/strong>Submitted on 10 Nov, 2020;<strong><br \/><a href=\"https:\/\/arxiv.org\/abs\/2011.04926\" target=\"_blank\" rel=\"noopener noreferrer\">arXiv:\u00a02011.04926<\/a><\/strong><\/li>\n<\/ul>\n<p><strong><\/strong><\/p>\n<ul>\n<li><strong>Approximation and convergence of GANs training: an SDE approach<br \/><\/strong>Haoyang Cao, Xin Guo<strong><br \/><\/strong>Submitted on 2 Jun, 2020;<strong><br \/><a href=\"https:\/\/arxiv.org\/abs\/2006.02047\" target=\"_blank\" rel=\"noopener noreferrer\">arXiv:\u00a02006.02047<\/a><\/strong><\/li>\n<\/ul>\n<p><strong><\/strong><\/p>\n<ul>\n<li><strong>Bridging Mode Connectivity in Loss Landscapes and Adversarial Robustness<br \/><\/strong>Pu Zhao, Pin-Yu Chen, Payel Das, Karthikeyan Natesan Ramamurthy, Xue Lin<strong><br \/><\/strong>Submitted on 30 Apr, 2020;<br \/><a href=\"https:\/\/arxiv.org\/abs\/2005.00060\" target=\"_blank\" rel=\"noopener noreferrer\"><strong>arXiv:\u00a0<span>2005.00060<\/span><\/strong><\/a><\/li>\n<\/ul>\n<p><strong><\/strong><\/p>\n<ul>\n<li><strong>Revisiting Loss Landscape for Adversarial Robustness<br \/><\/strong>Dongxian Wu, Yisen Wang, Shu-tao Xia<br \/> Submitted on 13 Apr, 2020;<br \/><a href=\"https:\/\/arxiv.org\/abs\/2004.05884\" target=\"_blank\" rel=\"noopener noreferrer\"><strong>arXiv:\u00a02004.05884<\/strong><\/a><\/li>\n<\/ul>\n<p><strong><\/strong><\/p>\n<ul>\n<li><strong>Understanding Global Loss Landscape of One-hidden-layer ReLU Neural Networks<br \/><\/strong>Bo Liu<br \/> Submitted on 12 Feb, 2020;<strong><br \/><\/strong><a href=\"https:\/\/arxiv.org\/abs\/2002.04763\" target=\"_blank\" rel=\"noopener noreferrer\"><strong>arXiv:\u00a02002.04763<\/strong><\/a><\/li>\n<\/ul>\n<p><strong><\/strong><\/p>\n<ul>\n<li><strong>Deep Ensembles: A Loss Landscape Perspective<\/strong><br \/>Stanislav Fort, Huiyi Hu, Balaji Lakshminarayanan<br \/> Submitted on 5 Dic, 2019;<br \/><strong><a href=\"https:\/\/arxiv.org\/abs\/1912.02757\" target=\"_blank\" rel=\"noopener noreferrer\">arXiv:\u00a01912.02757\n<p> <\/a><\/strong><\/p>\n<p><strong><a href=\"https:\/\/arxiv.org\/abs\/1912.02757\" target=\"_blank\" rel=\"noopener noreferrer\"><\/a><\/strong><strong><\/strong><\/p>\n<\/li>\n<li><strong>Emergent properties of the local geometry of neural loss landscapes<\/strong><br \/> Authors: Stanislav Fort, Surya Ganguli<br \/> Submitted on 14 Oct, 2019;<br \/> <strong><a href=\"https:\/\/arxiv.org\/abs\/1910.05929\" target=\"_blank\" rel=\"noopener noreferrer\">arXiv:1910.05929<br \/><\/a><\/strong><\/li>\n<\/ul>\n<ul>\n<li><strong>Loss Landscape Sightseeing with Multi-Point Optimization<\/strong><br \/> Authors: Ivan Skorokhodov, Mikhail Burtsev<br \/> Submitted on 9 Oct, 2019;<br \/> <strong><a href=\"https:\/\/arxiv.org\/abs\/1910.03867\" target=\"_blank\" rel=\"noopener noreferrer\">arXiv:1910.03867<br \/><\/a><\/strong><\/li>\n<\/ul>\n<ul>\n<li><strong>How noise affects the Hessian spectrum in overparameterized neural networks<\/strong><br \/> Authors: Mingwei Wei, David J Schwab<br \/> Submitted on 1 Oct, 2019;<br \/> <strong><a href=\"https:\/\/arxiv.org\/abs\/1910.00195\" target=\"_blank\" rel=\"noopener noreferrer\">arXiv:1910.00195<\/a><\/strong><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul>\n<li><strong>Visualizing and Understanding the Effectiveness of BERT<br \/> <\/strong>Authors:\u00a0<a href=\"https:\/\/arxiv.org\/search\/?searchtype=author&amp;query=Hao%2C+Y\">Yaru Hao<\/a>,\u00a0<a href=\"https:\/\/arxiv.org\/search\/?searchtype=author&amp;query=Dong%2C+L\">Li Dong<\/a>,\u00a0<a href=\"https:\/\/arxiv.org\/search\/?searchtype=author&amp;query=Wei%2C+F\">Furu Wei<\/a>,\u00a0<a href=\"https:\/\/arxiv.org\/search\/?searchtype=author&amp;query=Xu%2C+K\">Ke Xu<\/a><br \/> Submitted on 15 August, 2019;<strong><br \/> <a href=\"https:\/\/arxiv.org\/abs\/1908.05620\">arXiv:1908.05620<\/a><br \/> <\/strong><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul>\n<li><strong>The Difficulty of Training Sparse Neural Networks<\/strong><br \/> Authors: Utku Evci, Fabian Pedregosa, Aidan Gomez, Erich Elsen<br \/> Submitted on 25 June, 2019;<br \/> <strong><a href=\"https:\/\/arxiv.org\/abs\/1906.10732\" target=\"_blank\" rel=\"noopener noreferrer\">arXiv:1906.10732<\/a><\/strong><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul>\n<li><strong>Explaining Landscape Connectivity of Low-cost Solutions for Multilayer Nets<\/strong><br \/> Authors: Rohith Kuditipudi, Xiang Wang, Holden Lee, Yi Zhang, Zhiyuan Li, Wei Hu, Sanjeev Arora, Rong Ge<br \/> Submitted on 14 June, 2019;<br \/> <strong><a href=\"https:\/\/arxiv.org\/abs\/1906.06247\" target=\"_blank\" rel=\"noopener noreferrer\">arXiv:1906.06247<\/a><\/strong><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul>\n<li><strong>Large Scale Structure of Neural Network Loss Landscapes<br \/> <\/strong>Stanislav Fort, Stanislaw Jastrzebski<br \/> Submitted on 11 Jun 2019<strong><br \/> <a href=\"https:\/\/arxiv.org\/abs\/1906.04724\" target=\"_blank\" rel=\"noopener noreferrer\">arXiv:1906.04724<\/a><\/strong><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul>\n<li><strong>Understanding Generalization through Visualizations<br \/> <\/strong>W. Ronny Huang, Zeyad Emam, Micah Goldblum, Liam Fowl, Justin K. Terry, Furong Huang, Tom Goldstein<br \/> Submitted 7 June 2019<strong><br \/> <a href=\"https:\/\/arxiv.org\/abs\/1906.03291\" target=\"_blank\" rel=\"noopener noreferrer\">arXiv:1906.03291<\/a><br \/> <a href=\"https:\/\/arxiv.org\/abs\/1906.03291\">https:\/\/arxiv.org\/abs\/1906.03291<\/a><\/strong><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul>\n<li><strong>The Effect of Network Depth on the Optimization Landscape<br \/> <\/strong>Behrooz Ghorbani, Ying Xiao, Shankar Krishnan<br \/> Submitted 28 May 2019<br \/> <a href=\"https:\/\/openreview.net\/forum?id=SyxJ2y2qaE\" target=\"_blank\" rel=\"noopener noreferrer\"><b>Link<\/b><\/a><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul>\n<li><strong>Visualizing Loss Landscape of Deep Neural Networks\u2026..but can we Trust them?<br \/> <\/strong>Jae Duk Seo<br \/> May 5, 2019<br \/> <a href=\"https:\/\/towardsdatascience.com\/visualizing-loss-landscape-of-deep-neural-networks-but-can-we-trust-them-3d3ae0cff46e\" target=\"_blank\" rel=\"noopener noreferrer\"><b>Link<\/b><\/a><b><\/b><b><\/b><b><\/b><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul>\n<li><b>Negative eigenvalues of the Hessian in deep neural networks<br \/> <\/b>Guillaume Alain, Nicolas Le Roux, Pierre-Antoine Manzagol<br \/> Submitted on 6 Feb 2019<br \/> <a href=\"https:\/\/arxiv.org\/abs\/1902.02366\"><b>arXiv:1902.02366<\/b><\/a><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul>\n<li><strong>Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learning<br \/> <\/strong>Charles H. Martin, Michael W. Mahoney<br \/> Submitted on 2 Oct 2018<strong><br \/> <a href=\"https:\/\/arxiv.org\/abs\/1810.01075\" target=\"_blank\" rel=\"noopener noreferrer\">arXiv:1810.01075<\/a><\/strong><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul>\n<li><strong>On the loss landscape of a class of deep neural networks with no bad local valleys<br \/> <\/strong>Quynh Nguyen, Mahesh Chandra Mukkamala, Matthias Hein<br \/> Submitted on 27 Sep 2018<strong><br \/> <a href=\"https:\/\/arxiv.org\/abs\/1809.10749\" target=\"_blank\" rel=\"noopener noreferrer\">arXiv:1809.10749<\/a><\/strong><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul>\n<li><strong>The Goldilocks zone: Towards better understanding of neural network loss landscapes<br \/> <\/strong>Stanislav Fort, Adam Scherlis<br \/> Submitted on 6 Jul 2018 <strong><br \/> <a href=\"https:\/\/arxiv.org\/abs\/1807.02581\" target=\"_blank\" rel=\"noopener noreferrer\">arXiv:1807.02581<\/a><br \/> <\/strong><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul>\n<li><strong>The loss landscape of overparameterized neural networks<\/strong><br \/> Y Cooper<br \/> Submitted on 26 Apr 2018<br \/> <strong><a href=\"https:\/\/arxiv.org\/abs\/1804.10200\" target=\"_blank\" rel=\"noopener noreferrer\">arXiv:1804.10200<\/a><\/strong><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul>\n<li><strong>Measuring the Intrinsic Dimension of Objective Landscapes<\/strong><br \/> Chunyuan Li, Heerad Farkhoor, Rosanne Liu, Jason Yosinski<br \/> Submitted on 24 Apr 2018<br \/> <strong><a href=\"https:\/\/arxiv.org\/abs\/1804.08838\" target=\"_blank\" rel=\"noopener noreferrer\">arXiv:1804.08838<br \/> <\/a><\/strong><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul>\n<li><strong>A Mean Field View of the Landscape of Two-Layers Neural Networks<\/strong><br \/> Song Mei, Andrea Montanari, Phan-Minh Nguyen<br \/> Submitted on 18 Apr 2018<br \/> <strong><a href=\"https:\/\/arxiv.org\/abs\/1804.06561\" target=\"_blank\" rel=\"noopener noreferrer\">arXiv:1804.06561<br \/> <\/a><\/strong><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul>\n<li><strong>The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks<\/strong><br \/> Jonathan Frankle, Michael Carbin<br \/> Submitted on 9 Mar 2018<br \/> <strong><a href=\"https:\/\/arxiv.org\/abs\/1803.03635\" target=\"_blank\" rel=\"noopener noreferrer\">arXiv:1803.03635<\/a><\/strong><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul>\n<li><strong>Essentially No Barriers in Neural Network Energy Landscape<\/strong><br \/> Felix Draxler, Kambis Veschgini, Manfred Salmhofer, Fred A. Hamprecht<br \/> Submitted on 2 Mar 2018<br \/> <strong><a href=\"https:\/\/arxiv.org\/abs\/1803.00885\" target=\"_blank\" rel=\"noopener noreferrer\">arXiv:1803.00885<\/a><\/strong><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul>\n<li><strong>Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs<\/strong><br \/> Timur Garipov, Pavel Izmailov, Dmitrii Podoprikhin, Dmitry Vetrov, Andrew Gordon Wilson<br \/> Neural Information Processing Systems 2018<br \/> Submitted on 27 Feb 2018<br \/> <strong><a href=\"https:\/\/arxiv.org\/abs\/1802.10026\" target=\"_blank\" rel=\"noopener noreferrer\">arXiv:1802.10026<\/a><\/strong><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul>\n<li><strong>Visualizing the Loss Landscape of Neural Nets (A favourite of mine)\u00a0 *********<\/strong><br \/> Li, Hao and Xu, Zheng and Taylor, Gavin and Studer, Christoph and Goldstein, Tom<br \/> Neural Information Processing Systems 2018<br \/> Submitted on 28 Dec 2017<br \/> <strong><a href=\"https:\/\/arxiv.org\/abs\/1712.09913\" target=\"_blank\" rel=\"noopener noreferrer\">arXiv:1712.09913<\/a><\/strong><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul>\n<li><strong>Sharp Minima Can Generalize For Deep Nets<br \/> <\/strong>Laurent Dinh, Razvan Pascanu, Samy Bengio, Yoshua Bengio<br \/> Submitted on 28 March 2017<strong><br \/> <\/strong><a href=\"https:\/\/arxiv.org\/abs\/1703.04933\" target=\"_blank\" rel=\"noopener noreferrer\"><strong>arXiv:1703.04933<\/strong><\/a><\/li>\n<\/ul>\n<p>[\/et_pb_text][\/et_pb_column][\/et_pb_row][\/et_pb_section][et_pb_section fb_built=&#8221;1&#8243; admin_label=&#8221;Video&#8221; _builder_version=&#8221;3.22.7&#8243;][\/et_pb_section][et_pb_section fb_built=&#8221;1&#8243; admin_label=&#8221;Footer&#8221; _builder_version=&#8221;3.23.3&#8243; use_background_color_gradient=&#8221;on&#8221; background_color_gradient_start=&#8221;#eaeaea&#8221; background_color_gradient_end=&#8221;rgba(255,255,255,0)&#8221; background_color_gradient_direction=&#8221;90deg&#8221; background_color_gradient_end_position=&#8221;58%&#8221; background_color_gradient_overlays_image=&#8221;on&#8221; background_image=&#8221;https:\/\/losslandscape.com\/wp-content\/uploads\/2019\/09\/loss-landscape-research-17.jpg&#8221; background_position=&#8221;center_left&#8221; width=&#8221;100%&#8221; max_width=&#8221;100%&#8221; max_width_last_edited=&#8221;on|desktop&#8221; custom_padding=&#8221;0px||0px||true|false&#8221; locked=&#8221;off&#8221;][et_pb_row use_custom_gutter=&#8221;on&#8221; custom_padding=&#8221;0px||5vw||false&#8221; custom_margin=&#8221;||60px&#8221; make_equal=&#8221;on&#8221; custom_padding_last_edited=&#8221;on|phone&#8221; _builder_version=&#8221;3.22.7&#8243; background_image=&#8221;https:\/\/losslandscape.com\/wp-content\/uploads\/2019\/05\/calligraphy_68.png&#8221; background_position=&#8221;bottom_center&#8221; width=&#8221;100%&#8221; max_width=&#8221;100%&#8221; min_height=&#8221;27px&#8221;][et_pb_column type=&#8221;4_4&#8243; _builder_version=&#8221;3.22.5&#8243;][et_pb_text _builder_version=&#8221;3.22.7&#8243; text_font=&#8221;Allura||||||||&#8221; text_font_size=&#8221;28vw&#8221; text_line_height=&#8221;0.2em&#8221; header_3_font=&#8221;||||||||&#8221; text_orientation=&#8221;right&#8221; filter_opacity=&#8221;9%&#8221; locked=&#8221;off&#8221;]<\/p>\n<p>xyz<\/p>\n<p>[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row custom_padding=&#8221;|40px||40px||&#8221; custom_margin=&#8221;||60px&#8221; _builder_version=&#8221;3.22.7&#8243; background_size=&#8221;initial&#8221; background_position=&#8221;top_right&#8221; border_width_all=&#8221;1px&#8221; border_color_all=&#8221;rgba(0,0,0,0.12)&#8221; locked=&#8221;off&#8221;][et_pb_column type=&#8221;4_4&#8243; _builder_version=&#8221;3.22&#8243;][et_pb_image src=&#8221;https:\/\/losslandscape.com\/wp-content\/uploads\/2019\/05\/calligraphy_61.png&#8221; align=&#8221;right&#8221; _builder_version=&#8221;3.23&#8243; max_width=&#8221;200px&#8221; max_width_phone=&#8221;180px&#8221; max_width_last_edited=&#8221;on|desktop&#8221; custom_margin=&#8221;-130px|-8vw||&#8221; custom_margin_phone=&#8221;-150px||0px|&#8221; custom_margin_last_edited=&#8221;on|phone&#8221; always_center_on_mobile=&#8221;off&#8221; locked=&#8221;off&#8221;][\/et_pb_image][et_pb_text _builder_version=&#8221;3.23.3&#8243; header_font=&#8221;||||||||&#8221; header_2_font=&#8221;Muli|800||on|||||&#8221; header_2_font_size=&#8221;12px&#8221; header_2_letter_spacing=&#8221;0.7em&#8221; transform_scale_last_edited=&#8221;on|tablet&#8221; transform_translate=&#8221;-100px|100px&#8221; transform_translate_tablet=&#8221;0px|0px&#8221; transform_translate_last_edited=&#8221;on|tablet&#8221; transform_translate_linked=&#8221;off&#8221; transform_rotate=&#8221;0deg|0deg|270deg&#8221; transform_rotate_tablet=&#8221;0deg|0deg|0deg&#8221; transform_rotate_last_edited=&#8221;on|desktop&#8221; transform_skew_last_edited=&#8221;on|desktop&#8221; transform_origin=&#8221;0%|50%&#8221; transform_origin_last_edited=&#8221;on|desktop&#8221; transform_styles_last_edited=&#8221;on|tablet&#8221; max_width=&#8221;280px&#8221; custom_margin=&#8221;||-60px&#8221; custom_margin_tablet=&#8221;||20px&#8221; custom_margin_last_edited=&#8221;on|tablet&#8221; locked=&#8221;off&#8221;]<\/p>\n<h2>Go deep<\/h2>\n<p>[\/et_pb_text][et_pb_text _builder_version=&#8221;3.23.3&#8243; header_font=&#8221;||||||||&#8221; header_3_font=&#8221;Montserrat|600|||||||&#8221; header_3_font_size=&#8221;70px&#8221; header_3_line_height=&#8221;1.2em&#8221; use_background_color_gradient=&#8221;on&#8221; background_color_gradient_start=&#8221;#eaeaea&#8221; background_color_gradient_end=&#8221;rgba(255,255,255,0)&#8221; background_color_gradient_direction=&#8221;90deg&#8221; background_color_gradient_start_position=&#8221;20%&#8221; background_color_gradient_end_position=&#8221;50%&#8221; max_width=&#8221;700px&#8221; custom_margin=&#8221;||80px|-70px||&#8221; custom_margin_tablet=&#8221;0px||&#8221; custom_margin_phone=&#8221;|||-56px&#8221; custom_margin_last_edited=&#8221;on|phone&#8221; header_3_font_size_tablet=&#8221;50px&#8221; header_3_font_size_phone=&#8221;30px&#8221; header_3_font_size_last_edited=&#8221;on|phone&#8221;]<\/p>\n<h3>What lies within the landscape?<\/h3>\n<p>[\/et_pb_text][et_pb_button button_text=&#8221;About the Mission&#8221; button_alignment=&#8221;right&#8221; _builder_version=&#8221;3.23.3&#8243; custom_button=&#8221;on&#8221; button_text_size=&#8221;12px&#8221; button_text_color=&#8221;#ffffff&#8221; button_bg_color=&#8221;#000000&#8243; button_border_width=&#8221;0px&#8221; button_border_radius=&#8221;0px&#8221; button_letter_spacing=&#8221;4px&#8221; button_font=&#8221;Muli|600||on|||||&#8221; custom_margin=&#8221;||-60px|&#8221; custom_padding=&#8221;20px|30px|20px|30px|true|true&#8221;][\/et_pb_button][\/et_pb_column][\/et_pb_row][et_pb_row custom_padding=&#8221;||0px|||&#8221; custom_padding_last_edited=&#8221;off|tablet&#8221; _builder_version=&#8221;3.22.6&#8243; width=&#8221;100%&#8221; max_width=&#8221;100%&#8221; min_height=&#8221;27px&#8221; locked=&#8221;off&#8221;][et_pb_column type=&#8221;4_4&#8243; _builder_version=&#8221;3.22.5&#8243;][et_pb_image src=&#8221;https:\/\/losslandscape.com\/wp-content\/uploads\/2019\/05\/calligraphy_brushes_17.png&#8221; align_tablet=&#8221;center&#8221; align_last_edited=&#8221;on|desktop&#8221; _builder_version=&#8221;3.23&#8243;][\/et_pb_image][\/et_pb_column][\/et_pb_row][\/et_pb_section]<\/p>\n","protected":false},"excerpt":{"rendered":"<p>[et_pb_section fb_built=&#8221;1&#8243; admin_label=&#8221;Header&#8221; _builder_version=&#8221;3.23.3&#8243; use_background_color_gradient=&#8221;on&#8221; background_color_gradient_start=&#8221;#ffffff&#8221; background_color_gradient_end=&#8221;rgba(255,255,255,0)&#8221; background_color_gradient_direction=&#8221;188deg&#8221; background_color_gradient_end_position=&#8221;45%&#8221; background_color_gradient_overlays_image=&#8221;on&#8221; background_image=&#8221;https:\/\/losslandscape.com\/wp-content\/uploads\/2019\/10\/knowledgeback.jpg&#8221; width=&#8221;100%&#8221; max_width=&#8221;100%&#8221; max_width_last_edited=&#8221;on|desktop&#8221; custom_padding=&#8221;0px||0px||true|false&#8221;][et_pb_row use_custom_gutter=&#8221;on&#8221; custom_padding=&#8221;0px||||false&#8221; make_equal=&#8221;on&#8221; custom_padding_last_edited=&#8221;off|phone&#8221; _builder_version=&#8221;3.22.7&#8243; background_image=&#8221;https:\/\/losslandscape.com\/wp-content\/uploads\/2019\/05\/calligraphy_brushes_17.png&#8221; background_position=&#8221;bottom_center&#8221; width=&#8221;100%&#8221; max_width=&#8221;100%&#8221; min_height=&#8221;27px&#8221;][et_pb_column type=&#8221;4_4&#8243; _builder_version=&#8221;3.22.5&#8243;][et_pb_text _builder_version=&#8221;3.23.3&#8243; text_font=&#8221;Allura||||||||&#8221; text_font_size=&#8221;28vw&#8221; text_line_height=&#8221;0.2em&#8221; header_3_font=&#8221;||||||||&#8221; text_orientation=&#8221;right&#8221; filter_opacity=&#8221;9%&#8221;] Loss [\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row custom_padding_tablet=&#8221;17vw||17vw||false&#8221; custom_margin=&#8221;4vw|||&#8221; custom_padding_last_edited=&#8221;off|phone&#8221; padding_top_bottom_link_1=&#8221;true&#8221; padding_left_right_link_1=&#8221;true&#8221; padding_1_tablet=&#8221;60px||60px||true&#8221; padding_1_phone=&#8221;40px||40px||true&#8221; padding_1_last_edited=&#8221;off|phone&#8221; _builder_version=&#8221;3.22.7&#8243; background_position=&#8221;bottom_center&#8221; max_width=&#8221;80%&#8221; use_custom_width=&#8221;on&#8221; width_unit=&#8221;off&#8221; custom_width_percent=&#8221;100%&#8221;][et_pb_column type=&#8221;4_4&#8243; _builder_version=&#8221;3.12.2&#8243;][et_pb_text _builder_version=&#8221;3.23.3&#8243; text_font=&#8221;||||||||&#8221; text_text_color=&#8221;#ffffff&#8221; [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_et_pb_use_builder":"on","_et_pb_old_content":"","footnotes":""},"class_list":["post-53","page","type-page","status-publish","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v25.3 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Academic papers related to loss landscapes &amp; loss surfaces<\/title>\n<meta name=\"description\" content=\"Explore some of the latest academic papers that deal with the loss landscapes of deep learning optimization processes and related topics\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/losslandscape.com\/knowledge\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Academic papers related to loss landscapes &amp; loss surfaces\" \/>\n<meta property=\"og:description\" content=\"Explore some of the latest academic papers that deal with the loss landscapes of deep learning optimization processes and related topics\" \/>\n<meta property=\"og:url\" content=\"https:\/\/losslandscape.com\/knowledge\/\" \/>\n<meta property=\"og:site_name\" content=\"Landscape\" \/>\n<meta property=\"article:modified_time\" content=\"2021-03-13T22:55:37+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"9 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/losslandscape.com\/knowledge\/\",\"url\":\"https:\/\/losslandscape.com\/knowledge\/\",\"name\":\"Academic papers related to loss landscapes & loss surfaces\",\"isPartOf\":{\"@id\":\"https:\/\/losslandscape.com\/#website\"},\"datePublished\":\"2019-06-13T15:23:57+00:00\",\"dateModified\":\"2021-03-13T22:55:37+00:00\",\"description\":\"Explore some of the latest academic papers that deal with the loss landscapes of deep learning optimization processes and related topics\",\"breadcrumb\":{\"@id\":\"https:\/\/losslandscape.com\/knowledge\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/losslandscape.com\/knowledge\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/losslandscape.com\/knowledge\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/losslandscape.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Knowledge\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/losslandscape.com\/#website\",\"url\":\"https:\/\/losslandscape.com\/\",\"name\":\"Landscape\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/losslandscape.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Academic papers related to loss landscapes & loss surfaces","description":"Explore some of the latest academic papers that deal with the loss landscapes of deep learning optimization processes and related topics","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/losslandscape.com\/knowledge\/","og_locale":"en_US","og_type":"article","og_title":"Academic papers related to loss landscapes & loss surfaces","og_description":"Explore some of the latest academic papers that deal with the loss landscapes of deep learning optimization processes and related topics","og_url":"https:\/\/losslandscape.com\/knowledge\/","og_site_name":"Landscape","article_modified_time":"2021-03-13T22:55:37+00:00","twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"9 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/losslandscape.com\/knowledge\/","url":"https:\/\/losslandscape.com\/knowledge\/","name":"Academic papers related to loss landscapes & loss surfaces","isPartOf":{"@id":"https:\/\/losslandscape.com\/#website"},"datePublished":"2019-06-13T15:23:57+00:00","dateModified":"2021-03-13T22:55:37+00:00","description":"Explore some of the latest academic papers that deal with the loss landscapes of deep learning optimization processes and related topics","breadcrumb":{"@id":"https:\/\/losslandscape.com\/knowledge\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/losslandscape.com\/knowledge\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/losslandscape.com\/knowledge\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/losslandscape.com\/"},{"@type":"ListItem","position":2,"name":"Knowledge"}]},{"@type":"WebSite","@id":"https:\/\/losslandscape.com\/#website","url":"https:\/\/losslandscape.com\/","name":"Landscape","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/losslandscape.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/losslandscape.com\/wp-json\/wp\/v2\/pages\/53","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/losslandscape.com\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/losslandscape.com\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/losslandscape.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/losslandscape.com\/wp-json\/wp\/v2\/comments?post=53"}],"version-history":[{"count":49,"href":"https:\/\/losslandscape.com\/wp-json\/wp\/v2\/pages\/53\/revisions"}],"predecessor-version":[{"id":1368,"href":"https:\/\/losslandscape.com\/wp-json\/wp\/v2\/pages\/53\/revisions\/1368"}],"wp:attachment":[{"href":"https:\/\/losslandscape.com\/wp-json\/wp\/v2\/media?parent=53"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}