feat: Handle Amazon Q Agentic Chat artifacts in v2.9.5 and other build improvements

[enageshwari]

Description

feat: Handle Amazon Q Agentic Chat artifacts in v2.9.5 and other build improvements

• Download artifacts needed for Amazon Q agentic chat and set required permission for access. This make sure required artifacts are present for agentic chat to work even in private subnet scenarios. Though change is intended for SageMaker Unified Studio, SageMaker AI can also be benefitted out of this. More details in [1].
• Perform other build improvements for Python3 compatibility.
  • Migrated from conda.cli.python_api.run_command to subprocess.run
  • Using RequirementsSpec from conda.env.specs.requirements instead of conda.env.specs

Type of Change

• Image update - Bug fix
• Image update - New feature
• Image update - Breaking change
• SMD image build tool update
• Documentation update

Release Information

Does this change need to be included in patch version releases? By default, any pull requests will only be added to the next SMD image minor version release once they are merged in template folder. Only critical bug fix or security update should be applied to new patch versions of existed image minor versions.

• Yes (Critical bug fix or security update)
• No (New feature or non-critical change)
• N/A (Not an image update)

If yes, please explain why:
[Explain the criticality of this change and why it should be included in patch releases]
Private subnet use case is broken for SMUS JL users with SMD 2.9. More details in [1]

How Has This Been Tested?

• Existing unit tests passed
• New unit tests passed
• Functionality test:
  • Test SMD changes:
    - Followed instructions in CONTRIBUTING.md to test in 2.9.5 patch and 3..5.0 releases in local. Verified required artifacts are in right folder with right permissions and SMUS JL can use it. Check test results in [2].
    - Note: For patch versions, artifacts are copied from base version. Changes on Dockerfile are not automatically applied. Hence updated Dockerfile in 2.9.5 build artifacts with required changes and added required script for the build to go through.
  • Test if SMD changes will bring desired result to address [1]:
    - Steps followed for testing: Built image for 2.9.5 patch version -> created a container -> verified required artifacts are present with required permissions -> Reinstalled the SM GenAI JL extn with fix to pull from given dir. It worked. Test results in [2]

Checklist:

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have added tests that prove my fix is effective or that my feature works

Test Screenshots (if applicable):

Check test results in [2].

Related Issues

See [1]

References

[1] https://tiny.amazon.com/zmh6meka/tcorpamazP302
[2] https://tiny.amazon.com/2v58bfag/drivcorpamazfoldSMUSFail