RE: [EXT] Re: [PATCH v3 1/1] net: fec: add xdp and page pool statistics

[Shenwei Wang]

> -----Original Message-----
> From: Alexander Lobakin <alexandr.lobakin@xxxxxxxxx>
> Sent: Wednesday, November 16, 2022 8:34 AM
> To: Shenwei Wang <shenwei.wang@xxxxxxx>
> Cc: Alexander Lobakin <alexandr.lobakin@xxxxxxxxx>; Andrew Lunn
> <andrew@xxxxxxx>; David S. Miller <davem@xxxxxxxxxxxxx>; Eric Dumazet
> <edumazet@xxxxxxxxxx>; Jakub Kicinski <kuba@xxxxxxxxxx>; Paolo Abeni
> <pabeni@xxxxxxxxxx>; Alexei Starovoitov <ast@xxxxxxxxxx>; Daniel Borkmann
> <daniel@xxxxxxxxxxxxx>; Jesper Dangaard Brouer <hawk@xxxxxxxxxx>; John
> Fastabend <john.fastabend@xxxxxxxxx>; Wei Fang <wei.fang@xxxxxxx>;
> netdev@xxxxxxxxxxxxxxx; linux-kernel@xxxxxxxxxxxxxxx; imx@xxxxxxxxxxxxxxx;
> kernel test robot <lkp@xxxxxxxxx>
> Subject: [EXT] Re: [PATCH v3 1/1] net: fec: add xdp and page pool statistics
> 
> Caution: EXT Email
> 
> From: Shenwei Wang <shenwei.wang@xxxxxxx>
> Date: Mon, 14 Nov 2022 21:17:48 +0000
> 
> > > -----Original Message-----
> > > From: Alexander Lobakin <alexandr.lobakin@xxxxxxxxx>
> > > Sent: Monday, November 14, 2022 9:23 AM
> > > To: Shenwei Wang <shenwei.wang@xxxxxxx>
> > > Cc: Alexander Lobakin <alexandr.lobakin@xxxxxxxxx>; Andrew Lunn
> > > <andrew@xxxxxxx>; David S. Miller <davem@xxxxxxxxxxxxx>; Eric
> > > Dumazet <edumazet@xxxxxxxxxx>; Jakub Kicinski <kuba@xxxxxxxxxx>;
> > > Paolo Abeni <pabeni@xxxxxxxxxx>; Alexei Starovoitov
> > > <ast@xxxxxxxxxx>; Daniel Borkmann <daniel@xxxxxxxxxxxxx>; Jesper
> > > Dangaard Brouer <hawk@xxxxxxxxxx>; John Fastabend
> > > <john.fastabend@xxxxxxxxx>; Wei Fang <wei.fang@xxxxxxx>;
> > > netdev@xxxxxxxxxxxxxxx; linux-kernel@xxxxxxxxxxxxxxx;
> > > imx@xxxxxxxxxxxxxxx; kernel test robot <lkp@xxxxxxxxx>
> > > Subject: [EXT] Re: [PATCH v3 1/1] net: fec: add xdp and page pool
> > > statistics
> 
> [...]
> 
> > Did some testing with the atomic64_t counter, with the following codes
> > to update the u64 counter in the end of every NAPI poll cycle.
> >
> > @@ -1764,7 +1768,13 @@ fec_enet_rx_queue(struct net_device *ndev, int
> > budget, u16 queue_id)
> >
> >         if (xdp_result & FEC_ENET_XDP_REDIR)
> >                 xdp_do_flush_map();
> > +#if 1
> > +       if (xdp_prog) {
> > +               int i;
> > +               for(i = 0; i < XDP_STATS_TOTAL; i++)
> > +                       atomic64_add(xdp_stats[i], &rxq->stats[i]);
> > +       }
> > +#endif
> >         return pkt_received;
> >  }
> >
> > With the codes above, the testing result is below:
> > root@imx8qxpc0mek:~/bpf# ./xdpsock -i eth0
> >  sock0@eth0:0 rxdrop xdp-drv
> >                    pps            pkts           1.00
> > rx                 349399         1035008
> > tx                 0              0
> >
> >  sock0@eth0:0 rxdrop xdp-drv
> >                    pps            pkts           1.00
> > rx                 349407         1384640
> > tx                 0              0
> >
> > Without  the atomic_add codes above, the testing result is below:
> > root@imx8qxpc0mek:~/bpf# ./xdpsock -i eth0
> >  sock0@eth0:0 rxdrop xdp-drv
> >                    pps            pkts           1.00
> > rx                 350109         1989130
> > tx                 0              0
> >
> >  sock0@eth0:0 rxdrop xdp-drv
> >                    pps            pkts           1.00
> > rx                 350425         2339786
> > tx                 0              0
> >
> > And regarding the u32 counter solution, the testing result is below:
> >    root@imx8qxpc0mek:~/bpf# ./xdpsock -i eth0
> >      sock0@eth0:0 rxdrop xdp-drv
> >                        pps            pkts           1.00
> >     rx                 361347         2637796
> >     tx                 0              0
> >
> > There are about 10K pkts/s difference here. Do we really want the u64
> counters?
> 
> Where did those atomic64_t come from? u64_stats_t use either plain
> u64 for 32-bit platforms or local64_t for 64-bit ones. Take a look at [0] for the
> example of how x86_64 does this, it is far from atomic64_t.

The v5 patch has changed it to u64. 

Thanks,
Shenwei
> 
> >
> > Regards,
> > Shenwei
> >
> > >>
> > >> Thanks,
> > >> Shenwei
> > >>
> > >>>
> > >>>        Andrew
> > >
> > > Thanks,
> > > Olek
> 
> [0]
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Felixir.boo
> tlin.com%2Flinux%2Fv6.1-
> rc5%2Fsource%2Farch%2Fx86%2Finclude%2Fasm%2Flocal.h%23L31&amp;data
> =05%7C01%7Cshenwei.wang%40nxp.com%7Caf8d6634fcfc4d77e07308dac7dfa
> 4a8%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C638042060598040
> 501%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiL
> CJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=sWQ4Ts
> ayZvfs4pkCpC4tWvT51Cv89c4N6D5w2J%2FDC84%3D&amp;reserved=0
> 
> Thanks,
> Olek