Skip to content

Conversation

sundarshankar89
Copy link
Collaborator

@sundarshankar89 sundarshankar89 commented Sep 2, 2025

Changes

What does this PR do?

  • Introduces a new feature to establish a base configurator for Profiler Assessment within the project.

Relevant implementation details

Caveats/things to watch out for when reviewing:

Linked issues

Resolves #..

Functionality

  • added relevant user documentation
  • added new CLI command
  • modified existing command: databricks labs lakebridge ...
  • ... +add your own

Tests

  • manually tested
  • added unit tests
  • added integration tests

@sundarshankar89 sundarshankar89 requested a review from a team as a code owner September 2, 2025 12:28
@sundarshankar89 sundarshankar89 changed the title Add a new Feature to establish base configurator for Profiler Assessment Add a new feature to establish base configurator for Profiler Assessment Sep 2, 2025
Copy link

github-actions bot commented Sep 2, 2025

✅ 29/29 passed, 2 flaky, 1m23s total

Flaky tests:

  • 🤪 test_transpiles_informatica_with_sparksql (9.287s)
  • 🤪 test_transpile_sql_file (9.066s)

Running from acceptance #2397

@sundarshankar89 sundarshankar89 changed the title Add a new feature to establish base configurator for Profiler Assessment Introduce a new feature to establish base configurator for Profiler Assessment Sep 2, 2025
@sundarshankar89
Copy link
Collaborator Author

Merged #1940 in this

co-authored by: @goodwillpunning

logger.info(f"Creating a virtual environment for Python script execution: ${venv_dir}")
# Install dependencies in the virtual environment
# Define the paths to the virtual environment's Python and pip executables
if sys.platform == "win32":
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need to standardize this with databricks.labs.lakebridge.install.WheelInstaller._create_venv
please add a TODO for later

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • create_env
  • activate env
  • _install_dependencies
  • _run_python_script

are all generic and can be extracted out to a common utility class

"snowflake": SnowflakeConnector,
"mssql": MSSQLConnector,
"tsql": MSSQLConnector,
"synapse": MSSQLConnector,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this intentional?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes

@sundarshankar89 sundarshankar89 self-assigned this Sep 15, 2025
@sundarshankar89 sundarshankar89 added the feat/profiler Issues related to profilers label Sep 15, 2025
Copy link
Contributor

@m-abulazm m-abulazm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feat/profiler Issues related to profilers
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants