Looking for Feedback on Rewrite for Pytest in VS Code

[eleanorjboyd]

Hello! 👋
My name is Eleanor I am a new developer on the Python for VS Code team at Microsoft. I am working on a pytest rewrite and wanted some feedback from experienced pytest users about my logic to ensure VS Code will have the optimal pytest structure for discovery and run. Currently I am working on discovery, so discovering all the tests in a session and formatting then into nodes that will sit in our test tree view. example image below:

 
I have build a plugin that modifies behavior following test collection and will use mainly one flow for this discover test action. I have outlined the pseudocode below and want to know if I'm making best use of the pytest API. My biggest concern is having to start at the lowest level of the tree (the test_items) and build the file structure up from there to create a top down structure where the session instead refers to its direct children which are files/folders.
 

• Start with the session_node
• Iterate through all test_item in session.item:
  • Create a test_node
  • If the test_item_parent is a module:
    • Check to see if the test_item_parent_node is in my dictionary for file_and_folder_nodes
      • If not create
    • add test_item as child
  • If the test_item_parent is a class:
    • Check if test_item_class exists
      • If not create
    • Add test_item as child
    • Check if a node exists for the parent of the test_item_class
      • If not create
    • Add test_item_class as child
• Iterate through all modules in my dictionary of file_and_folder_nodes:
  • If the file name has multiple nested folders
    • Iterate through the folders and create new nodes for each folder
    • Set the previous folder as a child to the current folder (therefore nesting folders like their file structure)
  • Take the most recent file_node (this will be the one closest to the top of the file structure) and set it as a child to the session_node
• Return session_node

Thank you in advance! I look forward to collaborating with the open source community on this as I move forward 😃

[The-Compiler]

Is my understanding correct that you're basically trying to mirror the structure of pytest's own collection tree, which you can view by running pytest with --collect-only (minus the nested folders maybe)?

I'm a bit confused because you say "My biggest concern is having to start at the lowest level of the tree (the test_items) and build the file structure up from there", but your pseudocode seems to do the exact opposite, no?

If you haven't yet, you might also want to look at the code pytest itself uses for its --collect-only CLI flag:

pytest/src/_pytest/terminal.py

Lines 762 to 801 in 50b232b

	if self.config.getoption("collectonly"):
	if session.items:
	if self.config.option.verbose > -1:
	self._tw.line("")
	self._printcollecteditems(session.items)
	failed = self.stats.get("failed")
	if failed:
	self._tw.sep("!", "collection failures")
	for rep in failed:
	rep.toterminal(self._tw)
	def _printcollecteditems(self, items: Sequence[Item]) -> None:
	if self.config.option.verbose < 0:
	if self.config.option.verbose < -1:
	counts = Counter(item.nodeid.split("::", 1)[0] for item in items)
	for name, count in sorted(counts.items()):
	self._tw.line("%s: %d" % (name, count))
	else:
	for item in items:
	self._tw.line(item.nodeid)
	return
	stack: List[Node] = []
	indent = ""
	for item in items:
	needed_collectors = item.listchain()[1:] # strip root node
	while stack:
	if stack == needed_collectors[: len(stack)]:
	break
	stack.pop()
	for col in needed_collectors[len(stack) :]:
	stack.append(col)
	indent = (len(stack) - 1) * " "
	self._tw.line(f"{indent}{col}")
	if self.config.option.verbose >= 1:
	obj = getattr(col, "obj", None)
	doc = inspect.getdoc(obj) if obj else None
	if doc:
	for line in doc.splitlines():
	self._tw.line("{}{}".format(indent + " ", line))

[eleanorjboyd]

Hello @The-Compiler, thank you for getting back to me! So from my understanding the --collect-only flag allows the collection tree to be printed to the terminal and what it is printing is the session.items. Is this correctly? I am trying to access the test tree before it gets printed in order to convert each pytest object to a new object which is compatible with our VS Code testing API. Therefore would accessing session.items in the pytest_collection_finish have a similar result?

[The-Compiler]

Yep! The code snippet above is implemented in pytest_collection_finish, and indeed it does access session.items to print the tree with --collect-only. So indeed that approach seems reasonable!

[eleanorjboyd]

great thank you! I appreciate your feedback and expertise!