2 Matching Annotations
- Jun 2023
-
Local file Local file
-
A resource can map to the empty set, which allowsreferences to be made to a concept before any realization ofthat concept exist
This is a very useful but underutilized property. It allows you to e.g. announce in advance that a resource will exist at some point in the future, and thereby effectively receive "updates" to the linking document without requiring changes to the document itself.
-
- Mar 2023
-
www.semanticscholar.org www.semanticscholar.org
-
Over the past few years, many “efficient Trans-former” approaches have been proposed that re-duce the cost of the attention mechanism over longinputs (Child et al., 2019; Ainslie et al., 2020; Belt-agy et al., 2020; Zaheer et al., 2020; Wang et al.,2020; Tay et al., 2021; Guo et al., 2022). However,especially for larger models, the feedforward andprojection layers actually make up the majority ofthe computational burden and can render process-ing long inputs intractable
Recent improvements in transformers for long documents have focused on efficiencies in the attention mechanism but the feed-forward and projection layers are still expensive for long docs
-