Re: [PATCH] PCI/doe: Fix work struct declaration

[Dan Williams]

Ira Weiny wrote:
> On Wed, Nov 16, 2022 at 12:20:37PM -0600, Bjorn Helgaas wrote:
> > On Wed, Nov 16, 2022 at 11:09:39AM +0100, Lukas Wunner wrote:
> > > On Mon, Nov 14, 2022 at 05:19:43PM -0800, ira.weiny@xxxxxxxxx wrote:
> > > > From: Ira Weiny <ira.weiny@xxxxxxxxx>
> > > > 
> > > > The callers of pci_doe_submit_task() allocate the pci_doe_task on the
> > > > stack.  This causes the work structure to be allocated on the stack
> > > > without pci_doe_submit_task() knowing.  Work item initialization needs
> > > > to be done with either INIT_WORK_ONSTACK() or INIT_WORK() depending on
> > > > how the work item is allocated.
> > > > 
> > > > Jonathan suggested creating doe task allocation macros such as
> > > > DECLARE_CDAT_DOE_TASK_ONSTACK().[1]  The issue with this is the work
> > > > function is not known to the callers and must be initialized correctly.
> > > > 
> > > > A follow up suggestion was to have an internal 'pci_doe_work' item
> > > > allocated by pci_doe_submit_task().[2]  This requires an allocation which
> > > > could restrict the context where tasks are used.
> > > > 
> > > > Compromise with an intermediate step to initialize the task struct with
> > > > a new call pci_doe_init_task() which must be called prior to submit
> > > > task.
> > > > 
> > > > [1] https://lore.kernel.org/linux-cxl/20221014151045.24781-1-Jonathan.Cameron@xxxxxxxxxx/T/#m88a7f50dcce52f30c8bf5c3dcc06fa9843b54a2d
> > > > [2] https://lore.kernel.org/linux-cxl/20221014151045.24781-1-Jonathan.Cameron@xxxxxxxxxx/T/#m63c636c5135f304480370924f4d03c00357be667
> > > 
> > > We have object_is_on_stack(), included from <linux/sched/task_stack.h>.
> > > 
> > > So you could just autosense in pci_doe_submit_task() whether
> > > pci_doe_task is on the stack and call the appropriate INIT_WORK
> > > variant.
> > 
> > Nifty, I had no idea object_is_on_stack() existed, thank you!
> 
> Indeed!  Neither did I!  thanks!
> 
> > 
> > I wonder if there's an opportunity to use object_is_on_stack()
> > somewhere in the INIT_WORK() path to find usage mistakes.
> 
> I'm thinking we could make INIT_WORK do the right thing all the time.  Not sure
> what the overhead of object_is_on_stack() is.
> 
> > 
> > Adding it in pci_doe_submit_task() would add some complexity, so I'm
> > not sure whether it's worth adding it unless we actually have uses for
> > both cases.
> 
> I think if we don't do something we have to document that
> pci_doe_submit_task() only works with tasks on the stack.
> 
> I would rather just make pci_doe_submit_task() correct and not complicate the
> callers.  object_is_on_stack() can't be enough overhead to be worried about in
> this call path can it?
> 
> Actually after writing all that I wonder if we can't push the use of
> object_is_on_stack() into the debug code?  Something like below (completely
> untested)?  I think this could be pushed even further down but I'd like to get
> opinions before attempting a change which will have a wider blast radius.

This looks reasonable, but I would do it after and independently of
introducing the autosensing version of pci_doe_submit_task(). Then you
can pursue this line of thinking and come back to simplify
pci_doe_submit_task() if it indeed moves forward.