Improving Predictive Performance and Calibration by Weight Fusion in Semantic Segmentation
Authors:
Timo Sämann,
Ahmed Mostafa Hammam,
Andrei Bursuc,
Christoph Stiller,
Horst-Michael Groß
Abstract:
Averaging predictions of a deep ensemble of networks is apopular and effective method to improve predictive performance andcalibration in various benchmarks and Kaggle competitions. However, theruntime and training cost of deep ensembles grow linearly with the size ofthe ensemble, making them unsuitable for many applications. Averagingensemble weights instead of predictions circumvents this disadv…
▽ More
Averaging predictions of a deep ensemble of networks is apopular and effective method to improve predictive performance andcalibration in various benchmarks and Kaggle competitions. However, theruntime and training cost of deep ensembles grow linearly with the size ofthe ensemble, making them unsuitable for many applications. Averagingensemble weights instead of predictions circumvents this disadvantageduring inference and is typically applied to intermediate checkpoints ofa model to reduce training cost. Albeit effective, only few works haveimproved the understanding and the performance of weight averaging.Here, we revisit this approach and show that a simple weight fusion (WF)strategy can lead to a significantly improved predictive performance andcalibration. We describe what prerequisites the weights must meet interms of weight space, functional space and loss. Furthermore, we presenta new test method (called oracle test) to measure the functional spacebetween weights. We demonstrate the versatility of our WF strategy acrossstate of the art segmentation CNNs and Transformers as well as real worlddatasets such as BDD100K and Cityscapes. We compare WF with similarapproaches and show our superiority for in- and out-of-distribution datain terms of predictive performance and calibration.
△ Less
Submitted 8 November, 2022; v1 submitted 22 July, 2022;
originally announced July 2022.
Stacking of nanocrystalline graphene for Nano-Electro-Mechanical (NEM) actuator applications
Authors:
Kulothungan Jothiramalingam,
Marek E. Schmidt,
Muruganathan Manoharan,
Ahmed M. M. Hammam,
Hiroshi Mizuta
Abstract:
Graphene nano-electro-mechanical switches are promising components due to their excellent switching performance such as low pull-in voltage and low contact resistance. Mass fabrication with an appropriate counter electrode remains challenging. In this work, we report the stacking of nanocrystalline graphene (NCG) with a 70-nm dielectric separation layer. The buried NCG layer is contacted through t…
▽ More
Graphene nano-electro-mechanical switches are promising components due to their excellent switching performance such as low pull-in voltage and low contact resistance. Mass fabrication with an appropriate counter electrode remains challenging. In this work, we report the stacking of nanocrystalline graphene (NCG) with a 70-nm dielectric separation layer. The buried NCG layer is contacted through the formation of vias and acts as actuation electrode. After metallization, the top 7.5-nm thin NCG layer is patterned to form double-clamped beams, and the structure is released by hydrofluoric acid etching. By applying a voltage between the top and buried NCG layer, a step-like current increase is observed below 1.5 V, caused by the contact of the movable beam with the buried NCG. No pull-out is observed due to the thin sacrificial layer and high beam length, resulting in low mechanical restoring force. We discuss the possible applications of the NCG stacking approach to realize Nano-Electro-Mechanical (NEM) contact switches and advanced logical components such as a AND logic.
△ Less
Submitted 23 January, 2019;
originally announced January 2019.
