Using physical address as sk_buff data fragment
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty height:90px;width:728px;box-sizing:border-box;
Is it possible to map physical address as data fragment in sk_buff
?
I am working on Zynq Ultrascale+ platform (FPGA + ARM SOC). I have memory buffer mapped to physical address. The goal is to efficiently send that data over UDP. By efficiently I mean ZEROCOPY. What I am trying to do is to develop linux driver that would map that physical address into kernel memory and append it to sk_buff
as fragment.
I started with:
#define PACKET_LEN 1024
struct page *pag;
struct net_device *dev;
struct sk_buff *skb = NULL;
skb = alloc_skb(LL_RESERVED_SPACE(dev) + PACKET_LEN + ip_header_l +
udp_header_l, GFP_ATOMIC);
udp = skb_push(skb, udp_header_l);
//Fill up udp header
...
ip = skb_push(skb, ip_header_l);
//fill up ip header
...
dev_hard_header(skb, dev, ETH_P_IP, addr, myaddr, dev->addr_len);
skb->dev = dev;
//map page with data as fragment
skb_fill_page_desc(skb, 0, pag, 0, PACKET_LEN);
//send data
dev_queue_xmit(skb);
And as long as page is created by:
pagebuff = vmalloc(PACKET_LEN);
pag = vmalloc_to_page(pagebuff);
It all works fine. Packet gets send. Packet is send by two DMA transactions (Scatter Gather).
Going towards my goal I replaced vmalloc
ed page with:
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
membase = devm_ioremap_resource(&pdev->dev, res);
pag = virt_to_page(membase);
Physical address is 0xb0000000
and is mapped to virtual address 0xffffff800ad30000
page is at 0xffffffbf0025e280
.
After dev_queue_xmit
packet goes to network queue and ends up being mapped for DMA.
Problem arises when swiotlb_map_page
uses 0x00ad30000
as phys_addr
, which is different than original 0xb0000000
.virt_to_phys
is used in swiotlb_map_page
to calculate physical address and it basically takes lower 32 bits as phys address. Is there a different way to map memory region so it can be used as sk_buff
fragment?
As a temporary fix I created fake page like this:
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
pag = alloc_page(0); //create fake page
memset(pag, 0, sizeof(struct page));
pag->private = res->start;
And patched ethernet driver to use page private data as mapping address:
mapping = skb_frag_page(frag)->private;
if (mapping)
// printk("macb mapping override to %pn",mapping);
else
mapping = skb_frag_dma_map(&bp->pdev->dev, frag, offset, size, DMA_TO_DEVICE);
if (dma_mapping_error(&bp->pdev->dev, mapping))
goto dma_error;
With such a hack it all works. Data is filled with contents of 0xb0000000
. Although it works fine I really doubt it is the right way to do it. Nevertheless it shows there is no hardware limitation to do it. Does anyone know how to map that memory correctly?
P.S. I also tried to map physical address to fixed virtual address in such manner that swiotlb_map_page
would calculate correct address (and virt_to_phys
did), but it ended with "Unable to handle kernel paging request at virtual address" error.
membase = phys_to_virt(res->start);
i = ioremap_page_range(membase, membase + resource_size(res),
res->start, PAGE_KERNEL);
//I tried both
pag = phys_to_page(res->start);
pag = virt_to_page(membase);
Maybe I am looking for page at wrong address or maybe it is nonexistent.
Can anyone point me in the right direction. Is there a way to accomplish the goal without such a nasty hack?
c linux-kernel dma zynq ioremap
add a comment |
Is it possible to map physical address as data fragment in sk_buff
?
I am working on Zynq Ultrascale+ platform (FPGA + ARM SOC). I have memory buffer mapped to physical address. The goal is to efficiently send that data over UDP. By efficiently I mean ZEROCOPY. What I am trying to do is to develop linux driver that would map that physical address into kernel memory and append it to sk_buff
as fragment.
I started with:
#define PACKET_LEN 1024
struct page *pag;
struct net_device *dev;
struct sk_buff *skb = NULL;
skb = alloc_skb(LL_RESERVED_SPACE(dev) + PACKET_LEN + ip_header_l +
udp_header_l, GFP_ATOMIC);
udp = skb_push(skb, udp_header_l);
//Fill up udp header
...
ip = skb_push(skb, ip_header_l);
//fill up ip header
...
dev_hard_header(skb, dev, ETH_P_IP, addr, myaddr, dev->addr_len);
skb->dev = dev;
//map page with data as fragment
skb_fill_page_desc(skb, 0, pag, 0, PACKET_LEN);
//send data
dev_queue_xmit(skb);
And as long as page is created by:
pagebuff = vmalloc(PACKET_LEN);
pag = vmalloc_to_page(pagebuff);
It all works fine. Packet gets send. Packet is send by two DMA transactions (Scatter Gather).
Going towards my goal I replaced vmalloc
ed page with:
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
membase = devm_ioremap_resource(&pdev->dev, res);
pag = virt_to_page(membase);
Physical address is 0xb0000000
and is mapped to virtual address 0xffffff800ad30000
page is at 0xffffffbf0025e280
.
After dev_queue_xmit
packet goes to network queue and ends up being mapped for DMA.
Problem arises when swiotlb_map_page
uses 0x00ad30000
as phys_addr
, which is different than original 0xb0000000
.virt_to_phys
is used in swiotlb_map_page
to calculate physical address and it basically takes lower 32 bits as phys address. Is there a different way to map memory region so it can be used as sk_buff
fragment?
As a temporary fix I created fake page like this:
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
pag = alloc_page(0); //create fake page
memset(pag, 0, sizeof(struct page));
pag->private = res->start;
And patched ethernet driver to use page private data as mapping address:
mapping = skb_frag_page(frag)->private;
if (mapping)
// printk("macb mapping override to %pn",mapping);
else
mapping = skb_frag_dma_map(&bp->pdev->dev, frag, offset, size, DMA_TO_DEVICE);
if (dma_mapping_error(&bp->pdev->dev, mapping))
goto dma_error;
With such a hack it all works. Data is filled with contents of 0xb0000000
. Although it works fine I really doubt it is the right way to do it. Nevertheless it shows there is no hardware limitation to do it. Does anyone know how to map that memory correctly?
P.S. I also tried to map physical address to fixed virtual address in such manner that swiotlb_map_page
would calculate correct address (and virt_to_phys
did), but it ended with "Unable to handle kernel paging request at virtual address" error.
membase = phys_to_virt(res->start);
i = ioremap_page_range(membase, membase + resource_size(res),
res->start, PAGE_KERNEL);
//I tried both
pag = phys_to_page(res->start);
pag = virt_to_page(membase);
Maybe I am looking for page at wrong address or maybe it is nonexistent.
Can anyone point me in the right direction. Is there a way to accomplish the goal without such a nasty hack?
c linux-kernel dma zynq ioremap
add a comment |
Is it possible to map physical address as data fragment in sk_buff
?
I am working on Zynq Ultrascale+ platform (FPGA + ARM SOC). I have memory buffer mapped to physical address. The goal is to efficiently send that data over UDP. By efficiently I mean ZEROCOPY. What I am trying to do is to develop linux driver that would map that physical address into kernel memory and append it to sk_buff
as fragment.
I started with:
#define PACKET_LEN 1024
struct page *pag;
struct net_device *dev;
struct sk_buff *skb = NULL;
skb = alloc_skb(LL_RESERVED_SPACE(dev) + PACKET_LEN + ip_header_l +
udp_header_l, GFP_ATOMIC);
udp = skb_push(skb, udp_header_l);
//Fill up udp header
...
ip = skb_push(skb, ip_header_l);
//fill up ip header
...
dev_hard_header(skb, dev, ETH_P_IP, addr, myaddr, dev->addr_len);
skb->dev = dev;
//map page with data as fragment
skb_fill_page_desc(skb, 0, pag, 0, PACKET_LEN);
//send data
dev_queue_xmit(skb);
And as long as page is created by:
pagebuff = vmalloc(PACKET_LEN);
pag = vmalloc_to_page(pagebuff);
It all works fine. Packet gets send. Packet is send by two DMA transactions (Scatter Gather).
Going towards my goal I replaced vmalloc
ed page with:
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
membase = devm_ioremap_resource(&pdev->dev, res);
pag = virt_to_page(membase);
Physical address is 0xb0000000
and is mapped to virtual address 0xffffff800ad30000
page is at 0xffffffbf0025e280
.
After dev_queue_xmit
packet goes to network queue and ends up being mapped for DMA.
Problem arises when swiotlb_map_page
uses 0x00ad30000
as phys_addr
, which is different than original 0xb0000000
.virt_to_phys
is used in swiotlb_map_page
to calculate physical address and it basically takes lower 32 bits as phys address. Is there a different way to map memory region so it can be used as sk_buff
fragment?
As a temporary fix I created fake page like this:
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
pag = alloc_page(0); //create fake page
memset(pag, 0, sizeof(struct page));
pag->private = res->start;
And patched ethernet driver to use page private data as mapping address:
mapping = skb_frag_page(frag)->private;
if (mapping)
// printk("macb mapping override to %pn",mapping);
else
mapping = skb_frag_dma_map(&bp->pdev->dev, frag, offset, size, DMA_TO_DEVICE);
if (dma_mapping_error(&bp->pdev->dev, mapping))
goto dma_error;
With such a hack it all works. Data is filled with contents of 0xb0000000
. Although it works fine I really doubt it is the right way to do it. Nevertheless it shows there is no hardware limitation to do it. Does anyone know how to map that memory correctly?
P.S. I also tried to map physical address to fixed virtual address in such manner that swiotlb_map_page
would calculate correct address (and virt_to_phys
did), but it ended with "Unable to handle kernel paging request at virtual address" error.
membase = phys_to_virt(res->start);
i = ioremap_page_range(membase, membase + resource_size(res),
res->start, PAGE_KERNEL);
//I tried both
pag = phys_to_page(res->start);
pag = virt_to_page(membase);
Maybe I am looking for page at wrong address or maybe it is nonexistent.
Can anyone point me in the right direction. Is there a way to accomplish the goal without such a nasty hack?
c linux-kernel dma zynq ioremap
Is it possible to map physical address as data fragment in sk_buff
?
I am working on Zynq Ultrascale+ platform (FPGA + ARM SOC). I have memory buffer mapped to physical address. The goal is to efficiently send that data over UDP. By efficiently I mean ZEROCOPY. What I am trying to do is to develop linux driver that would map that physical address into kernel memory and append it to sk_buff
as fragment.
I started with:
#define PACKET_LEN 1024
struct page *pag;
struct net_device *dev;
struct sk_buff *skb = NULL;
skb = alloc_skb(LL_RESERVED_SPACE(dev) + PACKET_LEN + ip_header_l +
udp_header_l, GFP_ATOMIC);
udp = skb_push(skb, udp_header_l);
//Fill up udp header
...
ip = skb_push(skb, ip_header_l);
//fill up ip header
...
dev_hard_header(skb, dev, ETH_P_IP, addr, myaddr, dev->addr_len);
skb->dev = dev;
//map page with data as fragment
skb_fill_page_desc(skb, 0, pag, 0, PACKET_LEN);
//send data
dev_queue_xmit(skb);
And as long as page is created by:
pagebuff = vmalloc(PACKET_LEN);
pag = vmalloc_to_page(pagebuff);
It all works fine. Packet gets send. Packet is send by two DMA transactions (Scatter Gather).
Going towards my goal I replaced vmalloc
ed page with:
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
membase = devm_ioremap_resource(&pdev->dev, res);
pag = virt_to_page(membase);
Physical address is 0xb0000000
and is mapped to virtual address 0xffffff800ad30000
page is at 0xffffffbf0025e280
.
After dev_queue_xmit
packet goes to network queue and ends up being mapped for DMA.
Problem arises when swiotlb_map_page
uses 0x00ad30000
as phys_addr
, which is different than original 0xb0000000
.virt_to_phys
is used in swiotlb_map_page
to calculate physical address and it basically takes lower 32 bits as phys address. Is there a different way to map memory region so it can be used as sk_buff
fragment?
As a temporary fix I created fake page like this:
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
pag = alloc_page(0); //create fake page
memset(pag, 0, sizeof(struct page));
pag->private = res->start;
And patched ethernet driver to use page private data as mapping address:
mapping = skb_frag_page(frag)->private;
if (mapping)
// printk("macb mapping override to %pn",mapping);
else
mapping = skb_frag_dma_map(&bp->pdev->dev, frag, offset, size, DMA_TO_DEVICE);
if (dma_mapping_error(&bp->pdev->dev, mapping))
goto dma_error;
With such a hack it all works. Data is filled with contents of 0xb0000000
. Although it works fine I really doubt it is the right way to do it. Nevertheless it shows there is no hardware limitation to do it. Does anyone know how to map that memory correctly?
P.S. I also tried to map physical address to fixed virtual address in such manner that swiotlb_map_page
would calculate correct address (and virt_to_phys
did), but it ended with "Unable to handle kernel paging request at virtual address" error.
membase = phys_to_virt(res->start);
i = ioremap_page_range(membase, membase + resource_size(res),
res->start, PAGE_KERNEL);
//I tried both
pag = phys_to_page(res->start);
pag = virt_to_page(membase);
Maybe I am looking for page at wrong address or maybe it is nonexistent.
Can anyone point me in the right direction. Is there a way to accomplish the goal without such a nasty hack?
c linux-kernel dma zynq ioremap
c linux-kernel dma zynq ioremap
edited Nov 16 '18 at 14:45
red0ct
1,34631023
1,34631023
asked Nov 16 '18 at 12:23
modimomodimo
61
61
add a comment |
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53337862%2fusing-physical-address-as-sk-buff-data-fragment%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53337862%2fusing-physical-address-as-sk-buff-data-fragment%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown