From apw@shadowen.org Tue May 22 15:47:17 2007 Date: Wed, 23 May 2007 00:00:00 +0100 From: Andy Whitcroft To: linux-mm@kvack.org Cc: linux-arch@vger.kernel.org, Nick Piggin , Christoph Lameter , Mel Gorman , Andy Whitcroft Subject: [PATCH 4/8] x86_64: SPARSEMEM_VMEMMAP 2M page size support x86_64 uses 2M page table entries to map its 1-1 kernel space. We also implement the virtual memmap using 2M page table entries. So there is no additional runtime overhead over FLATMEM, initialisation is slightly more complex. As FLATMEM still references memory to obtain the mem_map pointer and SPARSEMEM_VMEMMAP uses a compile time constant, SPARSEMEM_VMEMMAP should be superior. With this SPARSEMEM becomes the most efficient way of handling virt_to_page, pfn_to_page and friends for UP, SMP and NUMA on x86_64. [apw@shadowen.org: code resplit, style fixups] From: Christoph Lameter Signed-off-by: Christoph Lameter Signed-off-by: Andy Whitcroft Acked-by: Mel Gorman --- Index: linux-2.6.22-rc2/Documentation/x86_64/mm.txt =================================================================== --- linux-2.6.22-rc2.orig/Documentation/x86_64/mm.txt 2007-05-18 21:06:17.000000000 -0700 +++ linux-2.6.22-rc2/Documentation/x86_64/mm.txt 2007-05-22 17:24:07.000000000 -0700 @@ -9,6 +9,7 @@ ffff800000000000 - ffff80ffffffffff (=40 ffff810000000000 - ffffc0ffffffffff (=46 bits) direct mapping of all phys. memory ffffc10000000000 - ffffc1ffffffffff (=40 bits) hole ffffc20000000000 - ffffe1ffffffffff (=45 bits) vmalloc/ioremap space +ffffe20000000000 - ffffe2ffffffffff (=40 bits) virtual memory map (1TB) ... unused hole ... ffffffff80000000 - ffffffff82800000 (=40 MB) kernel text mapping, from phys 0 ... unused hole ... Index: linux-2.6.22-rc2/arch/x86_64/Kconfig =================================================================== --- linux-2.6.22-rc2.orig/arch/x86_64/Kconfig 2007-05-18 21:06:17.000000000 -0700 +++ linux-2.6.22-rc2/arch/x86_64/Kconfig 2007-05-22 17:24:07.000000000 -0700 @@ -392,6 +392,14 @@ config ARCH_SPARSEMEM_ENABLE def_bool y depends on (NUMA || EXPERIMENTAL) +config SPARSEMEM_VMEMMAP + def_bool y + depends on SPARSEMEM + +config ARCH_POPULATES_SPARSEMEM_VMEMMAP_PMD + def_bool y + depends on SPARSEMEM_VMEMMAP + config ARCH_MEMORY_PROBE def_bool y depends on MEMORY_HOTPLUG Index: linux-2.6.22-rc2/arch/x86_64/mm/init.c =================================================================== --- linux-2.6.22-rc2.orig/arch/x86_64/mm/init.c 2007-05-18 21:06:17.000000000 -0700 +++ linux-2.6.22-rc2/arch/x86_64/mm/init.c 2007-05-22 17:24:07.000000000 -0700 @@ -761,3 +761,33 @@ int in_gate_area_no_task(unsigned long a { return (addr >= VSYSCALL_START) && (addr < VSYSCALL_END); } + +#ifdef CONFIG_ARCH_POPULATES_SPARSEMEM_VMEMMAP_PMD +/* + * Initialise the sparsemem vmemmap using huge-pages at the PMD level. + */ +int __meminit vmemmap_populate_pmd(pud_t *pud, unsigned long addr, + unsigned long end, int node) +{ + pmd_t *pmd; + + for (pmd = pmd_offset(pud, addr); addr < end; + pmd++, addr += PMD_SIZE) + if (pmd_none(*pmd)) { + pte_t entry; + void *p = vmemmap_alloc_block(PMD_SIZE, node); + if (!p) + return -ENOMEM; + + entry = pfn_pte(__pa(p) >> PAGE_SHIFT, PAGE_KERNEL); + mk_pte_huge(entry); + set_pmd(pmd, __pmd(pte_val(entry))); + + printk(KERN_DEBUG " [%lx-%lx] PMD ->%p on node %d\n", + addr, addr + PMD_SIZE - 1, p, node); + } else + vmemmap_verify((pte_t *)pmd, node, + pmd_addr_end(addr, end), end); + return 0; +} +#endif Index: linux-2.6.22-rc2/include/asm-x86_64/page.h =================================================================== --- linux-2.6.22-rc2.orig/include/asm-x86_64/page.h 2007-05-18 21:06:17.000000000 -0700 +++ linux-2.6.22-rc2/include/asm-x86_64/page.h 2007-05-22 17:24:07.000000000 -0700 @@ -133,6 +133,7 @@ extern unsigned long __phys_addr(unsigne VM_READ | VM_WRITE | VM_MAYREAD | VM_MAYWRITE | VM_MAYEXEC) #define __HAVE_ARCH_GATE_AREA 1 +#define vmemmap ((struct page *)VMEMMAP_START) #include #include Index: linux-2.6.22-rc2/include/asm-x86_64/pgtable.h =================================================================== --- linux-2.6.22-rc2.orig/include/asm-x86_64/pgtable.h 2007-05-18 21:06:17.000000000 -0700 +++ linux-2.6.22-rc2/include/asm-x86_64/pgtable.h 2007-05-22 17:24:07.000000000 -0700 @@ -137,6 +137,7 @@ static inline pte_t ptep_get_and_clear_f #define MAXMEM _AC(0x3fffffffffff, UL) #define VMALLOC_START _AC(0xffffc20000000000, UL) #define VMALLOC_END _AC(0xffffe1ffffffffff, UL) +#define VMEMMAP_START _AC(0xffffe20000000000, UL) #define MODULES_VADDR _AC(0xffffffff88000000, UL) #define MODULES_END _AC(0xfffffffffff00000, UL) #define MODULES_LEN (MODULES_END - MODULES_VADDR)