VSCode Datalayer: Implement new lexical editor and update datalayer notebook #405

goanpeca · 2025-09-16T04:44:37Z

…ive notebooks

- Add specific webpack rule for .whl (Python wheel) files - Ensures wheel files are handled as asset/resource type - Fixes build errors on Windows CI for pyodide kernel dependencies - Maintains compatibility with existing pypi resource handling

Copilot

Pull Request Overview

This PR implements a new lexical editor and updates the datalayer notebook functionality for the VS Code extension. The changes introduce a comprehensive theme system that integrates with VS Code's theming, a new lexical editor for rich text editing, and enhanced notebook functionality with improved runtime management.

Enhanced theme system with VS Code integration and dynamic theme switching
New lexical editor with rich text editing capabilities and collaboration support
Improved notebook functionality with better runtime selection and Datalayer integration
Updated build configuration to support both notebook and lexical editor webviews

Reviewed Changes

Copilot reviewed 68 out of 69 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
packages/vscode/webview/utils.ts	Added module documentation and new saveToBytes utility function
packages/vscode/webview/theme/types/index.ts	Comprehensive theme type definitions for extensible theme system
packages/vscode/webview/theme/providers/VSCodeThemeProvider.ts	VS Code theme provider with color extraction and syntax highlighting
packages/vscode/webview/theme/providers/BaseThemeProvider.ts	Base theme provider implementation with common functionality
packages/vscode/webview/theme/mapping/UniversalColorMapper.ts	Universal color mapping system between different theme systems
packages/vscode/webview/theme/index.ts	Main theme system exports and API
packages/vscode/webview/theme/components/ThemedLoader.tsx	VS Code themed loading component
packages/vscode/webview/theme/components/EnhancedJupyterReactTheme.tsx	Enhanced theme wrapper with VS Code integration
packages/vscode/webview/theme/components/CodeMirrorThemeInjector.tsx	CodeMirror theme injection for syntax highlighting
packages/vscode/webview/theme/codemirror/patchCodeMirror.ts	CodeMirror patching for VS Code syntax highlighting
packages/vscode/webview/theme/codemirror/createVSCodeTheme.ts	CodeMirror theme creation from VS Code colors
packages/vscode/webview/theme/codemirror/VSCodeCodeMirrorTheme.ts	Complete CodeMirror theme matching VS Code
packages/vscode/webview/theme/README.md	Comprehensive documentation for the theme system
packages/vscode/webview/serviceManager.ts	Enhanced service manager with better documentation
packages/vscode/webview/mockServiceManager.ts	Mock service manager for Datalayer notebooks
packages/vscode/webview/messageHandler.ts	Enhanced message handling with better logging
packages/vscode/webview/main.ts	Simplified entry point with better documentation
packages/vscode/webview/lexicalWebview.tsx	Main lexical editor webview implementation
packages/vscode/webview/NotebookVSCode.tsx	Completely rewritten notebook component with enhanced functionality
packages/vscode/webview/NotebookToolbar.tsx	VS Code styled toolbar for notebook operations
packages/vscode/webview/LexicalToolbar.tsx	Comprehensive toolbar for lexical editor formatting
packages/vscode/webview/LexicalEditor.tsx	Main lexical editor with VS Code theme integration
packages/vscode/webview/LexicalEditor.css	Complete CSS styling for lexical editor with VS Code theme variables
packages/vscode/webpack.config.js	Enhanced webpack configuration with CodeMirror deduplication
packages/vscode/tsconfig.json	Updated TypeScript configuration
packages/vscode/tsconfig.doc.json	New TypeScript configuration for documentation
packages/vscode/src/util.ts	Enhanced utility functions with documentation
packages/vscode/src/spaces/spacesTreeProvider.ts	Tree provider for Datalayer spaces view
packages/vscode/src/spaces/spacerApiService.ts	Comprehensive API service for Datalayer platform
packages/vscode/src/spaces/spaceItem.ts	Data models for spaces and documents
packages/vscode/src/spaces/documentBridge.ts	Document lifecycle management bridge

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-09-24T14:37:44Z

packages/vscode/webview/theme/providers/VSCodeThemeProvider.ts

+    console.log(
+      '[VSCodeThemeProvider] Initializing with color mode:',
+      colorMode,
+    );


Consider using a more structured logging approach or reducing the verbosity of console.log statements in production code. Multiple console.log statements throughout the file may impact performance and clutter logs.

Suggested change

console.log(

'[VSCodeThemeProvider] Initializing with color mode:',

colorMode,

);

// Removed verbose console.log to avoid cluttering production logs.

Copilot · 2025-09-24T14:37:45Z

packages/vscode/webview/theme/providers/VSCodeThemeProvider.ts

+    console.log('[VSCodeThemeProvider] Detected VS Code colors:', {
+      count: this._vscodeColors.size,
+      sample: Array.from(this._vscodeColors.entries()).slice(0, 5),
+      editorBg: this._vscodeColors.get('--vscode-editor-background'),
+      editorFg: this._vscodeColors.get('--vscode-editor-foreground'),
+    });


The console.log statement creates a potentially large object for debugging. Consider using a debug flag or environment variable to control verbose logging to avoid performance overhead in production.

Suggested change

console.log('[VSCodeThemeProvider] Detected VS Code colors:', {

count: this._vscodeColors.size,

sample: Array.from(this._vscodeColors.entries()).slice(0, 5),

editorBg: this._vscodeColors.get('--vscode-editor-background'),

editorFg: this._vscodeColors.get('--vscode-editor-foreground'),

});

if (

(typeof window !== 'undefined' && (window as any).DEBUG) ||

(typeof process !== 'undefined' && process.env && process.env.DEBUG)

) {

console.log('[VSCodeThemeProvider] Detected VS Code colors:', {

count: this._vscodeColors.size,

sample: Array.from(this._vscodeColors.entries()).slice(0, 5),

editorBg: this._vscodeColors.get('--vscode-editor-background'),

editorFg: this._vscodeColors.get('--vscode-editor-foreground'),

});

}

Copilot · 2025-09-24T14:37:45Z

packages/vscode/webview/theme/codemirror/patchCodeMirror.ts

+  const isDark = getColor('--vscode-editor-background', '#ffffff')
+    .toLowerCase()
+    .includes('2');
+


The dark theme detection logic is fragile and unreliable. Checking if a color string includes '2' is not a proper way to determine if a theme is dark. This could misclassify colors like '#ffffff' or '#f2f2f2'.

Suggested change

const isDark = getColor('--vscode-editor-background', '#ffffff')

.toLowerCase()

.includes('2');

// Helper to determine if a color is "dark" based on luminance

function isColorDark(hexColor: string): boolean {

// Remove leading '#' if present

hexColor = hexColor.trim();

if (hexColor.startsWith('#')) {

hexColor = hexColor.slice(1);

}

// Support short hex (#fff) and long hex (#ffffff)

if (hexColor.length === 3) {

hexColor = hexColor[0] + hexColor[0] +

hexColor[1] + hexColor[1] +

hexColor[2] + hexColor[2];

}

if (hexColor.length !== 6) {

// Fallback: treat as light

return false;

}

const r = parseInt(hexColor.slice(0, 2), 16);

const g = parseInt(hexColor.slice(2, 4), 16);

const b = parseInt(hexColor.slice(4, 6), 16);

// Relative luminance formula

const luminance = (0.2126 * r + 0.7152 * g + 0.0722 * b) / 255;

// Threshold: < 0.5 is dark

return luminance < 0.5;

}

const editorBg = getColor('--vscode-editor-background', '#ffffff');

const isDark = isColorDark(editorBg);

Copilot · 2025-09-24T14:37:45Z

packages/vscode/webview/NotebookVSCode.tsx

+window.debugNotebook = function () {
+  const all = document.querySelectorAll('*');
+  const results: any[] = [];
+
+  all.forEach((el: Element) => {
+    const style = getComputedStyle(el);
+    const bg = style.backgroundColor;
+
+    // Check for black or near-black backgrounds
+    if (bg && bg !== 'rgba(0, 0, 0, 0)' && bg !== 'transparent') {
+      const match = bg.match(/rgba?\((\d+),\s*(\d+),\s*(\d+)/);
+      if (match) {
+        const [_, r, g, b] = match;
+        if (parseInt(r) < 50 && parseInt(g) < 50 && parseInt(b) < 50) {
+          results.push({
+            element: el,
+            tag: el.tagName,
+            id: el.id,
+            classes: el.className,
+            backgroundColor: bg,
+            parent: el.parentElement?.tagName,
+            parentId: el.parentElement?.id,
+            parentClasses: el.parentElement?.className,
+          });
+        }
+      }
+    }
+  });
+
+  console.log('Dark elements found:', results);
+  results.forEach(r => {
+    console.log(
+      `${r.tag}#${r.id || 'no-id'}.${r.classes || 'no-class'} = ${r.backgroundColor}`,
+    );
+  });
+  return results;
+};
+
+// Log that the debug function is available
+console.log(
+  '[NotebookVSCode] Debug function installed on window.debugNotebook',
+);


Global debug functions should not be added to the window object in production code. Consider removing this debug functionality or wrapping it in a development-only check.

Suggested change

window.debugNotebook = function () {

const all = document.querySelectorAll('*');

const results: any[] = [];

all.forEach((el: Element) => {

const style = getComputedStyle(el);

const bg = style.backgroundColor;

// Check for black or near-black backgrounds

if (bg && bg !== 'rgba(0, 0, 0, 0)' && bg !== 'transparent') {

const match = bg.match(/rgba?\((\d+),\s*(\d+),\s*(\d+)/);

if (match) {

const [_, r, g, b] = match;

if (parseInt(r) < 50 && parseInt(g) < 50 && parseInt(b) < 50) {

results.push({

element: el,

tag: el.tagName,

id: el.id,

classes: el.className,

backgroundColor: bg,

parent: el.parentElement?.tagName,

parentId: el.parentElement?.id,

parentClasses: el.parentElement?.className,

});

}

}

}

});

console.log('Dark elements found:', results);

results.forEach(r => {

console.log(

`${r.tag}#${r.id || 'no-id'}.${r.classes || 'no-class'} = ${r.backgroundColor}`,

);

});

return results;

};

// Log that the debug function is available

console.log(

'[NotebookVSCode] Debug function installed on window.debugNotebook',

);

if (process.env.NODE_ENV !== 'production') {

window.debugNotebook = function () {

const all = document.querySelectorAll('*');

const results: any[] = [];

all.forEach((el: Element) => {

const style = getComputedStyle(el);

const bg = style.backgroundColor;

// Check for black or near-black backgrounds

if (bg && bg !== 'rgba(0, 0, 0, 0)' && bg !== 'transparent') {

const match = bg.match(/rgba?\((\d+),\s*(\d+),\s*(\d+)/);

if (match) {

const [_, r, g, b] = match;

if (parseInt(r) < 50 && parseInt(g) < 50 && parseInt(b) < 50) {

results.push({

element: el,

tag: el.tagName,

id: el.id,

classes: el.className,

backgroundColor: bg,

parent: el.parentElement?.tagName,

parentId: el.parentElement?.id,

parentClasses: el.parentElement?.className,

});

}

}

}

});

console.log('Dark elements found:', results);

results.forEach(r => {

console.log(

`${r.tag}#${r.id || 'no-id'}.${r.classes || 'no-class'} = ${r.backgroundColor}`,

);

});

return results;

};

// Log that the debug function is available

console.log(

'[NotebookVSCode] Debug function installed on window.debugNotebook',

);

}

Copilot · 2025-09-24T14:37:46Z

packages/vscode/webview/LexicalToolbar.tsx

+        if ($isHeadingNode(element) && element.getTag() === headingTag) {
+          // If already this heading type, convert to paragraph
+          const paragraph = element.replace(
+            $createHeadingNode('h1').replace($createHeadingNode('h1')),
+            true,
+          );
+        } else {


The paragraph conversion logic is incorrect. The code creates a new h1 node and immediately replaces it with another h1 node, then uses that result to replace the current element. This doesn't actually convert to a paragraph.

Copilot · 2025-09-24T14:37:46Z

packages/vscode/webpack.config.js

+      // Rule for pyodide kernel wheel files
+      {
+        test: /\.whl$/,
+        type: 'asset/resource',
+        generator: {
+          filename: 'pypi/[name][ext]',
+        },
+      },


The duplicate pyodide/pypi rules (lines 162-169 and 171-177) handle different file patterns but both use similar configurations. Consider consolidating these rules or adding comments to clarify the distinction.

echarles

LGTM Thx @goanpeca

goanpeca self-assigned this Sep 16, 2025

goanpeca changed the title ~~Implement login and logout functionality~~ VSCode Datalayer: Implement new lexical editor and update login and space tre view Sep 16, 2025

goanpeca changed the title ~~VSCode Datalayer: Implement new lexical editor and update login and space tre view~~ VSCode Datalayer: Implement new lexical editor and update login and space tree view Sep 16, 2025

goanpeca added 11 commits September 17, 2025 15:48

Implement login and logout functionality

13e0117

Add tree view support to browse datalayer spaces

4bb50b5

Add documentation and typedoc and update runtime handling

27c00a7

Add support for lexical documents!

b7f2ee6

Improve handling of remote lexicals

d172a1a

WIP: Current state before collaboration branch

cf4e15a

Reorganize code and add support for using datalayer runtimes with nat…

f628c63

…ive notebooks

Reorganize code and add support for using datalayer runtimes with nat…

0eec459

…ive notebooks

Enable collaboration on Lexicals

c21e443

Improve UI for lexicals

f017783

Test notebook

a56fb80

goanpeca force-pushed the enh/vscode-updates branch from cad78b1 to a56fb80 Compare September 17, 2025 20:52

goanpeca added 8 commits September 17, 2025 15:57

Testing

55620e0

Update spaces operations

c426f56

Enable saving and loading for datalayer notebooks

c72d87a

Add vscode theme handling

bd7ea68

More theming and resizing fixes!

f40ea94

More theme and notebook fixes!

2a90891

Add CI workflows

010ea08

Fixes for win ci and license headers

067d08f

goanpeca force-pushed the enh/vscode-updates branch 2 times, most recently from da2d1b1 to 81caea7 Compare September 22, 2025 01:55

License headers

910a075

goanpeca force-pushed the enh/vscode-updates branch from 81caea7 to 910a075 Compare September 22, 2025 01:56

goanpeca added 2 commits September 21, 2025 22:08

Update documentation and checks

8cfc2b7

goanpeca changed the title ~~VSCode Datalayer: Implement new lexical editor and update login and space tree view~~ VSCode Datalayer: Implement new lexical editor and update datalayer notebook Sep 22, 2025

Add runtime selector

edc1a59

goanpeca marked this pull request as ready for review September 24, 2025 14:34

Copilot AI review requested due to automatic review settings September 24, 2025 14:34

Copilot AI reviewed Sep 24, 2025

View reviewed changes

echarles approved these changes Sep 24, 2025

View reviewed changes

echarles merged commit a6e961c into datalayer:main Sep 24, 2025
12 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

VSCode Datalayer: Implement new lexical editor and update datalayer notebook #405

VSCode Datalayer: Implement new lexical editor and update datalayer notebook #405

Uh oh!

goanpeca commented Sep 16, 2025 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Sep 24, 2025

Uh oh!

Copilot AI Sep 24, 2025

Uh oh!

Copilot AI Sep 24, 2025

Uh oh!

Copilot AI Sep 24, 2025

Uh oh!

Copilot AI Sep 24, 2025

Uh oh!

Copilot AI Sep 24, 2025

Uh oh!

echarles left a comment

Uh oh!

Uh oh!

Uh oh!

-  const isDark = getColor('--vscode-editor-background', '#ffffff')
-    .toLowerCase()
-    .includes('2');
+  // Helper to determine if a color is "dark" based on luminance
+  function isColorDark(hexColor: string): boolean {
+    // Remove leading '#' if present
+    hexColor = hexColor.trim();
+    if (hexColor.startsWith('#')) {
+      hexColor = hexColor.slice(1);
+    }
+    // Support short hex (#fff) and long hex (#ffffff)
+    if (hexColor.length === 3) {
+      hexColor = hexColor[0] + hexColor[0] +
+                 hexColor[1] + hexColor[1] +
+                 hexColor[2] + hexColor[2];
+    }
+    if (hexColor.length !== 6) {
+      // Fallback: treat as light
+      return false;
+    }
+    const r = parseInt(hexColor.slice(0, 2), 16);
+    const g = parseInt(hexColor.slice(2, 4), 16);
+    const b = parseInt(hexColor.slice(4, 6), 16);
+    // Relative luminance formula
+    const luminance = (0.2126 * r + 0.7152 * g + 0.0722 * b) / 255;
+    // Threshold: < 0.5 is dark
+    return luminance < 0.5;
+  }
+  const editorBg = getColor('--vscode-editor-background', '#ffffff');
+  const isDark = isColorDark(editorBg);

Uh oh!

VSCode Datalayer: Implement new lexical editor and update datalayer notebook #405

VSCode Datalayer: Implement new lexical editor and update datalayer notebook #405

Uh oh!

Conversation

goanpeca commented Sep 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

echarles left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

goanpeca commented Sep 16, 2025 •

edited

Loading