Leveraging Memories

Lumi AI offers users the ability to moderate its behaviour in responses as means of real-time "training"/influence by selectively reinforcing positive and negative responses. This moderation is managed by forming what are termed "Memories".

What are "Memories"?

Each memory is an example instance that Lumi AI will use for a particular Knowledge Base as a reference to either good (positive) or bad (negative) behaviour from a previous response/workflow.

Memories are created by providing "feedback" in chat (see further below on Generating Memories). Here are some example scenarios and why/how a user may choose to provide feedback:

• Positive Reinforcement Example

  • Scenario: Lumi AI correctly joined multiple chained tables to apply a complex formula/metric described in the Knowledge Base context.

  • Reason: This will be used to influence the same pattern in future prompts surrounding the metric within the correct contextual application in other cases.

• Negative Reinforcement Example

  • Scenario: Lumi AI filtered on a target item id but in the wrong field. Thus the response was "inaccurate" by virtue of incorrect field selection.

  • Reason: It is as valuable to understand what is valid as invalid, thus indicating the wrong field was used will inform future examples of what to avoid doing, promoting the right behaviour instead.

Note that at the moment, memories strictly influence SQL Code generation only. Memories will have no effect on the insights, graph formatting, or other aspects of a response.

Generating Memories

Memories go through three phases: proposal, endorsement, and invalidation.

1. Proposal

Memories are generated when you provide "feedback" to a response in Chat. This creates a "proposed" memory in the Knowledge Base for Contributors to apply.

A memory will have no effect until it has been reviewed.

2. Endorsement

In the Knowledge Base Memories tab, Contributors can view proposed memories and either "commit" or "reject" the memory.

• If the memory correctly identifies a positive or negative example, the Contributor should "commit" the memory, which (on Save of the KB), will make it available for all.

• If the memory proposed incorrectly identifies a positive or negative example (i.e., a false positive or false negative, as in a faulty or inaccurate responsive that has been dubbed positive or an ideal response that has been dubbed negative), the Contributor should "reject" the memory.

Once all proposals have been committed or rejected, the Contributor can save the Knowledge Base and make memories immediately effective in Chat and Boards.

2.1 Special Consideration: Volume of Similar Memories

Each memory will indicate how many "Similar Memories" it has (if any). When reviewing Proposed Memories, it is worthwhile to evaluate this number:

• 0 -- No similar memories means it's a worthwhile candidate for endorsement, subject to all other considerations that may apply.

• 1 to 5 -- Having a few memories is better than 0 or 1, and having a mix of positive and negative or other combinations further improves performance via memories, so a small volume is worthwhile to complement with additional records. Completely redundant records though are likely low value-add.

• 5+ -- Beyond 5 memories (largely an arbitrary suggested limit, it may be worthwhile to have more) it is unlikely that additional endorsement/committal of memories has positive impact. For performance optimization, a subset of memories are only ever applied, with a preference for mixing positive and negative variations, thus only the most relevant memories will be used. In some instances, slightly lower relevance but high distinctiveness in a memory may be more valuable, thus it is recommended to keep the volume low and distinct.

3. Invalidation

The final lifecycle stage of a memory is "invalidation". If a dependency changes (see below), this means that the memory may no longer apply. For example, the Knowledge Base may originally indicate that average sales officially refers to the last 3 months of sales when no time period is specified, but the Knowledge Base may update and evolve such that the definition changes to 6 months, in which case positive reinforcement with 3 months are now invalid and should no longer be used.

Once invalidated, a memory will cease being used and cannot be recovered.

Memories have strong influence and are thus highly impactful on responses. For this reason, invalidation is sensitive and absolute, to minimize the complexity in finely addressing compatibility of changes and minimizing overall Contributor effort.

Understanding "Dependencies"

While memories are incredibly powerful for quickly and effectively tuning or dialling-in behaviour for a Knowledge Base, they can also potentially hold back a Knowledge Base as it evolves. For this reason, each memory is associated with its own "Dependencies", as hinted earlier.

A dependency is simply a static/immutable reference to an existing context element in the Knowledge Base. This can be:

• A table or field, its source, and definition

• A model join/relationship

• A metric, detail, or other "business context" element

When any of these items are modified, this causes that particular named dependency to change or be removed, thus causing "invalidation" (see earlier life cycle notes).

Note that in rare examples it is possible for the addition of context and possible future dependencies to adversely affect an otherwise valid, committed memory. In these cases, the existing memory should be reviewed and rejected.

Listing Memories

All Knowledge Base members are free to view memories in all their states -- proposed, committed, rejected, and invalidated -- as well as all their relevant details in the Memories tab.

For more information and how to interact, see the Memories section of the Knowledge Base.

Observing Memories Usage

Memories are observable in our "Blocks" framework, during context isolation. Here you can see the examples (positive and negative) that are selected for influence in parallel with relevant context pulled for the response.

"Blocks" are available in private preview only, please stay tuned for the full release!

Future Functionality: Users will be able to disable or force-select from memories in the future.