What company leaders face, he said, is not an innovation problem but an impatience problem.

Research has shown that involving workers' perspectives in the design of workplace technologies promotes sustainable improvements in productivity and well-being.

The organizations that get this right won't be the ones that just automated the most tasks. They'll be the ones that figured out when the human should act, when the agent should act, and how the handoff between them works.

The OpenAI team recently published a fantastic piece detailing the creation of their own internal data agent. It's a transparent detail of a very detailed and elegant implementation – but points to the long journey required to get there.

You have to have people that have the ability to rethink the workflow at a scale that AI can execute, versus at a scale that humans can execute.

You have to have people that have the ability to rethink the workflow at a scale that AI can execute, versus at a scale that humans can execute.

highlights are incorporated into the input text with special markups, <extra_id_1> and <extra_id_2>, marking the beginning and end of each highlighted span, respectively. In our configuration, we set the maximum input length to 4096 and the maximum target length to 400. A greedy decoding strategy was used in order to optimize the decoding speed.

Our approach locates the longest common subsequence (LCS) between the lemmas of each input sentence and each summary sentence, followed by several heuristics to filter out irrelevant LCSs

For the summarization model, we used a BARTlarge model (Lewis et al., 2019) trained on the CNN/Daily Mail dataset (Hermann et al., 2015), selected for its noticeable popularity.

For the initial auto-consolidation, we deploy an available Controlled Text Reduction model (Slobodkin et al., 2023), which is a Flan-T5large model (Chung et al., 2022), finetuned on the highlights-focused CTR dataset.

we deploy the ExtractiveSummarizer model from the TransformerSum library. The model, a RoBERTabase (Liu et al., 2019) trained on the CNN/DailyMail summarization dataset (Hermann et al., 2015), operates as a binary classifier.

errors encountered during API calls are handled in two ways: handle within our system or delegate to users. We handle known LLM API errors that can be solved by user-side intervention. This would be in cases such as a Timeout or RateLimitError in OpenAI models

errors such as APIConnectionError in OpenAI models occur because of an issue with the LLM API server itself and requires intervention from OpenAI.

While MEGAnno+ is designed to support any open-source LLM or commercial LLM APIs, in this work, we only demonstrate OpenAI Completion models for clarity and brevity.

Data Model MEGAnno+ extends MEGAnno's data model where data Record, Label, Annotation, Metadata (e.g., text embedding or confidence score) are persisted in the service database along with the task Schema.

To implement our system as an extension to MEGAnno (Zhang et al., 2022), an in-notebook exploratory annotation tool.

MEGAnno+ is designed to provide a convenient and robust workflow for users to utilize LLMs in text annotation. To use our tool, users operate within their Jupyter notebook (Kluyver et al., 2016) with the MEGAnno+ client installed.