Skip to content

Mem-World (full title not recorded)

Session: Not recorded. The slide photo was taken at 11:25 am on Mon Nov 9, shortly after the DreamZero talk (Oral Session 2, Talk 1). Speaker(s): Paper: Not yet found. The name comes from the slide footer, "Mem-World."

Paper summary

Not yet found. Everything below is from one slide photo (slide 6 of 15).

Talk notes

Method: let future actions determine which history to retrieve.

  • Wrist-view-centred 4D surfel memory module:
  • A surfel-based memory index covers every past timestep.
  • The planned actions give a new end-effector pose and a new wrist camera pose.
  • From that pose, the memory renders a surfel map for each past timestep, with depth, normals, timestep, and the manipulated object.
  • A scorer rates each past timestep on geometry, task relevance, and recency. Top-K plus non-maximum suppression (NMS) then picks a diverse set of history frames.
  • The slide's summary: "Actions → average future wrist pose → render each historical timestep separately."
  • Multi-view action-conditioned world model:
  • Its input is the retrieved, geometry-related history frames, followed by noised future frames.
  • Each frame has P = v × h × w tokens, where v is the number of camera views and h and w are the latent height and width.
  • Each of the N blocks has an action-conditioned spatial transformer followed by a 1D temporal transformer.

Mem-World action-driven memory retrieval

Questions / discussion

  • How does this compare with MemoryVAM, the other Day 1 memory work?