clarifciaiton on --override and swapping

[cburroughs]

#2425 describes " swapping" an entire dependency. I may have such a case.

I'm trying to work with a proprietary wheel [1] that uses the deprecated "Obsoletes" and "Provides" https://packaging.python.org/en/latest/specifications/core-metadata/ metadata to try to "replace" another wheel . Said proprietary wheel also includes forked code at the same import path as the package it purports to "Provide", so having both in a lockfile can quickly lead to runtime problems. I am aware that pip does not support this metadata and the scheme obviously doesn't work and creates a big mess.

I'm exploring options in a lockfile to either:

• Always replace the open source wheel with the proprietary one.
• Maybe wash my hands of this and do the opposite.

I can see from the docs and examples how to use --exclude to exclude a wheel (never use urllib3), and how to use --override to change the dependencies in the graph (always use urllib3==1.21). But I'm not seeing how to do a full swap ("always use urllib3-my-fork instead of urllib3) as suggested by #2425. Was the full swap a current capability, or possible future enhancement?

(In the above case I could create an empty dummy internal urllib3 wheel that just Requires-Dist to urllib3-my-fork, but I didn't think that was what was implied.)

My preferred solution would be for the large well financed company to not be so... gratuitously messy with their packaging. My intent is to clarify the current capability, not ask for free feature work to circumvent proprietary problems.

[1] https://pypi.org/project/databricks-connect/15.4.9/#files

[jsirois]

1st stop should always be -h:

--override OVERRIDDEN
                        Specifies a transitive requirement to override when
                        resolving. Any distribution requirement in the PEX's
                        resolve that matches the override project name is
                        replaced with the given override requirement. This
                        option can be used multiple times. (default: [])

So --override matches on project name, meaning you can override a project to come from a different version or source (say VCS), but you cannot change the project name itself.

That code is here (Note in particular line 40 where the project name comes from the requirement itself):

pex/pex/dependency_configuration.py

Lines 25 to 53 in 722a0b6

	@attr.s(frozen=True)
	class DependencyConfiguration(object):
	@classmethod
	def create(
	cls,
	excluded=(), # type: Iterable[Union[str, Requirement]]
	overridden=(), # type: Iterable[Union[str, Requirement]]
	):
	# type: (...) -> DependencyConfiguration
	overridden_projects = defaultdict(
	OrderedSet
	) # type: DefaultDict[ProjectName, OrderedSet[Requirement]]
	for o in overridden:
	override = o if isinstance(o, Requirement) else Requirement.parse(o)
	overridden_projects[override.project_name].add(override)
	return cls(
	excluded=tuple(
	OrderedSet(
	req if isinstance(req, Requirement) else Requirement.parse(req)
	for req in excluded
	)
	),
	overridden={
	project_name: tuple(overrides)
	for project_name, overrides in overridden_projects.items()
	},
	)

I think a full swap could be retrofitted unambiguously though, say using syntax like: --override <project name>=<req>. If you want that @cburroughs please file an issue and I'll get it knocked out. But consider a fair trade answering my question on your recentish Pip issue about PyPy / possible universal lock resolves (pypa/pip#13546 (comment)).

[jsirois]

Alright, @cburroughs - too slow. The feature now exists. Please take a look at #2894 and, even though you did not agree to the trade, please also take a look at pypa/pip#13546 (comment) (if this a case of a Pex universal lock use of Pip, I really don't want the Pip maintainers on any goose chase).

[jsirois]

Hopefully https://github.com/pex-tool/pex/releases/tag/v2.55.0 gets you down the road.

[cburroughs]

Thank you.

I've revisited pypa/pip/issues/13546 and will take #2894 for a spin tomorrow.

[jsirois]

@cburroughs has this worked for you / can this discussion be marked answered? If not I'd like to know and get this sorted instead of having it linger.