GIT 169e1a2a8a789fa84254695ec6a56fc410bb19a9 git+ssh://master.kernel.org/pub/scm/linux/kernel/git/jejb/scsi-misc-2.6.git commit 169e1a2a8a789fa84254695ec6a56fc410bb19a9 Author: Andrew Morton Date: Tue Apr 18 21:09:08 2006 -0700 [SCSI] scsi_lib.c: fix warning in scsi_kmap_atomic_sg drivers/scsi/scsi_lib.c: In function `scsi_kmap_atomic_sg': drivers/scsi/scsi_lib.c:2394: warning: unsigned int format, different type arg (arg 3) drivers/scsi/scsi_lib.c:2394: warning: unsigned int format, different type arg (arg 4) Signed-off-by: Andrew Morton Signed-off-by: James Bottomley commit c5f2e6404c65e8380c9ba80a7d58a27d2642743b Author: akpm@osdl.org Date: Sat Apr 15 00:30:24 2006 -0700 [SCSI] scsi_scan.c: fix compile warnings drivers/scsi/scsi_scan.c: In function `scsi_probe_and_add_lun': drivers/scsi/scsi_scan.c:926: warning: unused variable `vend' drivers/scsi/scsi_scan.c:926: warning: unused variable `mod' drivers/scsi/scsi_scan.c: At top level: drivers/scsi/scsi_scan.c:829: warning: `scsi_inq_str' defined but not used Fix those, tighten up the (somewhat poorly-designed) logging macro and fix some coding-style warts. Signed-off-by: Andrew Morton Signed-off-by: James Bottomley commit cdb8c2a6d848deb9eeefffff42974478fbb51b8c Author: Guennadi Liakhovetski Date: Sun Apr 2 21:57:43 2006 +0200 [SCSI] dc395x: dynamically map scatter-gather for PIO The current dc395x driver uses PIO to transfer up to 4 bytes which do not get transferred by DMA (under unclear circumstances). For this the driver uses page_address() which is broken on highmem. Apart from this the actual calculation of the virtual address is wrong (even without highmem). So, e.g., for reading it reads bytes from the driver to a wrong address and returns wrong data, I guess, for writing it would just output random data to the device. The proper fix, as suggested by many, is to dynamically map data using kmap_atomic(page, KM_BIO_SRC_IRQ) / kunmap_atomic(virt). The reason why it has not been done until now, although I've done some preliminary patches more than a year ago was that nobody interested in fixing this problem was able to reliably reproduce it. Now it changed - with the help from Sebastian Frei (CC'ed) I was able to trigger the PIO path. Thus, I was also able to test and debug it. There are 4 cases when PIO is used in dc395x - data-in / -out with and without scatter-gather. I was able to reproduce and test only data-in with and without SG. So, the data-out path is still untested, but it is also somewhat simpler than the data-in. Fredrik Roubert (also CC'ed) also had PIO triggering on his system, and in his case it was data-out without SG. It would be great if he could test the attached patch on his system, but even if he cannot, I would still request to apply the patch and just wait if anybody cries... Implementation: I put 2 new functions in scsi_lib.c and their declarations in scsi_cmnd.h. I exported them without _GPL, although, I don't feel strongly about that - not many drivers are likely to use them. But there is at least one more - I want to use them in tmscsim.c. Whether these are the right files for the functions and their declarations - not sure either. Actually, they are not scsi-specific, so, might go somewhere around other scattergather magic? They are not platform specific either, and most SG functions are defined under arch/*/... As these issues were discussed previously there were some more routines suggested to manipulate scattergather buffers, I think, some of them were needed around crypto code... So, might be a common place reasonable, like lib/scattergather.c? I am open here. Signed-off-by: James Bottomley commit 4c021dd136c0ad524e6d117296beafad2bf570c0 Author: FUJITA Tomonori Date: Fri Apr 7 19:10:03 2006 +0900 [SCSI] ibmvscsi: convert kmalloc + memset to kcalloc Convert kmalloc + memset to kcalloc in ibmvscsi Signed-off-by: FUJITA Tomonori Acked-by: Dave Boutcher Signed-off-by: James Bottomley commit 5bb0b55a3283369f1cd8ac76a6d8bda8e7a77055 Author: Mike Christie Date: Thu Apr 6 21:26:46 2006 -0500 [SCSI] iscsi: convert iscsi tcp to libiscsi This just converts iscsi_tcp to the lib Signed-off-by: Mike Christie Signed-off-by: James Bottomley commit 7996a778ff8c717cb1a7a294475c59cc8f1e9fb8 Author: Mike Christie Date: Thu Apr 6 21:13:41 2006 -0500 [SCSI] iscsi: add libiscsi There is a lot of code duplcited between iscsi_tcp and the upcoming iscsi_iser driver. This patch puts the duplicated code in a lib. There is more code to move around but this takes care of the basics. For iscsi_offload if they use the lib we will probably move some things around. For example in the queuecommand we will not assume that the LLD wants to do queue_work, but it is better to handle that later when we know for sure what iscsi_offload looks like (we could probably do this for iscsi_iser though to). Ideally I would like to get the iscsi_transports modules to a place where all they really have to do is put data on the wire, but how to do that will hopefully be more clear when we see other modules like iscsi_offload. Or maybe iscsi_offload will not use the lib and it will just be iscsi_iser and iscsi_tcp and maybe the iscsi_tcp_tgt if that is allowed in mainline. Signed-off-by: Mike Christie Signed-off-by: James Bottomley commit 30a6c65236f9d26e3325cae468f330b833a3878c Author: Mike Christie Date: Thu Apr 6 21:13:39 2006 -0500 [SCSI] iscsi: fix up iscsi eh The current iscsi_tcp eh is not nicely setup for dm-multipath and performs some extra task management functions when they are not needed. The attached patch: - Fixes the TMF issues. If a session is rebuilt then we do not send aborts. - Fixes the problem where if the host reset fired, we would return SUCCESS even though we had not really done anything yet. This ends up causing problem with scsi_error.c's TUR. - If someone has turned on the userspace nop daemon code to try and detect network problems before the scsi command timeout we can now drop and clean up the session before the scsi command timesout and fires the eh speeding up the time it takes for a command to go from one patch to another. For network problems we fail the command with DID_BUS_BUSY so if failfast is set scsi_decide_disposition fails the command up to dm for it to try on another path. - And we had to add some basic iscsi session block code. Previously if we were trying to repair a session we would retrun a MLQUEUE code in the queuecommand. This worked but it was not the most efficient or pretty thing to do since it would take a while to relogin to the target. For iscsi_tcp/open-iscsi a lot of the iscsi error handler is in userspace the block code is pretty bare. We will be adding to that for qla4xxx. Signed-off-by: Mike Christie Signed-off-by: James Bottomley commit fd7255f51a13ea915099c7e488001dfbbeb05104 Author: Mike Christie Date: Thu Apr 6 21:13:36 2006 -0500 [SCSI] iscsi: add sysfs attrs for uspace sync up For iscsi boot when going from initramfs to the real root we need to stop the userpsace iscsi daemon. To later restart it iscsid needs to be able to rebuild itself and part of that process is matching a session running the kernel with the iscsid representation. To do this the attached patch adds several required iscsi values. If the LLD does not provide them becuase, login is done in userspace, then the transport class and userspace set ths up for the LLD. Signed-off-by: Mike Christie Signed-off-by: James Bottomley commit b5c7a12dc29ae0990d9e867749bdd717a3160325 Author: Mike Christie Date: Thu Apr 6 21:13:33 2006 -0500 [SCSI] iscsi: rm kernel iscsi handles usage for session and connection from hare@suse.de and michaelc@cs.wisc.edu hw iscsi like qla4xxx does not allocate a host per session and for userspace it is difficult to restart iscsid using the "iscsi handles" for the session and connection, so this patch just has the class or userspace allocate the id for the session and connection. Note: this breaks userspace and requires users to upgrade to the newest open-iscsi tools. Sorry about his but open-iscsi is still too new to say we have a stable user-kernel api and we were not good nough designers to know that other hw iscsi drivers and iscsid itself would need such changes. Actually we sorta did but at the time we did not have the HW available to us so we could only guess. Luckily, the only tools hooking into the class are the open-iscsi ones or other tools like iscsitart hook into the open-iscsi engine from userspace or prgroams like anaconda call our tools so they are not affected. Signed-off-by: Mike Christie Signed-off-by: James Bottomley commit 13f7e5acc8b329080672c13f05f252ace5b79825 Author: Kurt Garloff Date: Mon Apr 3 15:20:08 2006 +0200 [SCSI] BLIST_ATTACH_PQ3 flags Some devices report a peripheral qualifier of 3 for LUN 0; with the original code, we would still try a REPORT_LUNS scan (if SCSI level is >= 3 or if we have the BLIST_REPORTLUNS2 passed in), but NOT any sequential scan. Also, the device at LUN 0 (which is not connected according to the PQ) is not registered with the OS. Unfortunately, SANs exist that are SCSI-2 and do NOT support REPORT_LUNS, but report a unknown device with PQ 3 on LUN 0. We still need to scan them, and most probably we even need BLIST_SPARSELUN (and BLIST_LARGELUN). See the bug reference for an infamous example. This is patch 3/3: 3. Implement the blacklist flag BLIST_ATTACH_PQ3 that makes the scsi scanning code register PQ3 devices and continues scanning; only sg will attach thanks to scsi_bus_match(). Signed-off-by: Kurt Garloff Signed-off-by: James Bottomley commit 6c7154c97e20c0ea28547240dc86731c0cee1b2f Author: Kurt Garloff Date: Mon Apr 3 15:18:35 2006 +0200 [SCSI] Better log messages for PQ3 devs Some devices report a peripheral qualifier of 3 for LUN 0; with the original code, we would still try a REPORT_LUNS scan (if SCSI level is >= 3 or if we have the BLIST_REPORTLUNS2 passed in), but NOT any sequential scan. Also, the device at LUN 0 (which is not connected according to the PQ) is not registered with the OS. Unfortunately, SANs exist that are SCSI-2 and do NOT support REPORT_LUNS, but report a unknown device with PQ 3 on LUN 0. We still need to scan them, and most probably we even need BLIST_SPARSELUN (and BLIST_LARGELUN). See the bug reference for an infamous example. This patch 2/3: If a PQ3 device is found, log a message that describes the device (INQUIRY DATA and C:B:T:U tuple) and make a suggestion for blacklisting it. Signed-off-by: Kurt Garloff Signed-off-by: James Bottomley commit 4186ab1973758190916703eb8889ebe8002c5c8f Author: Kurt Garloff Date: Mon Apr 3 15:16:48 2006 +0200 [SCSI] Try LUN 1 and use bflags Some devices report a peripheral qualifier of 3 for LUN 0; with the original code, we would still try a REPORT_LUNS scan (if SCSI level is >= 3 or if we have the BLIST_REPORTLUNS2 passed in), but NOT any sequential scan. Also, the device at LUN 0 (which is not connected according to the PQ) is not registered with the OS. Unfortunately, SANs exist that are SCSI-2 and do NOT support REPORT_LUNS, but report a unknown device with PQ 3 on LUN 0. We still need to scan them, and most probably we even need BLIST_SPARSELUN (and BLIST_LARGELUN). See the bug reference for an infamous example. This is patch 1/3: If we end up in sequential scan, at least try LUN 1 for devices that reported a PQ of 3 for LUN 0. Also return blacklist flags, even for PQ3 devices. Signed-off-by: Kurt Garloff Signed-off-by: James Bottomley --- Signed-off-by: Andrew Morton --- drivers/scsi/Makefile | 2 drivers/scsi/dc395x.c | 221 + drivers/scsi/ibmvscsi/ibmvscsi.c | 3 drivers/scsi/iscsi_tcp.c | 2883 ++++++++------------------ drivers/scsi/iscsi_tcp.h | 179 - drivers/scsi/libiscsi.c | 1697 +++++++++++++++ drivers/scsi/scsi_devinfo.c | 2 drivers/scsi/scsi_lib.c | 58 drivers/scsi/scsi_logging.h | 8 drivers/scsi/scsi_scan.c | 64 drivers/scsi/scsi_transport_iscsi.c | 622 +++-- include/scsi/iscsi_if.h | 106 include/scsi/libiscsi.h | 286 ++ include/scsi/scsi_cmnd.h | 4 include/scsi/scsi_devinfo.h | 1 include/scsi/scsi_transport_iscsi.h | 107 16 files changed, 3747 insertions(+), 2496 deletions(-) diff -puN drivers/scsi/dc395x.c~git-scsi-misc drivers/scsi/dc395x.c --- devel/drivers/scsi/dc395x.c~git-scsi-misc 2006-04-30 00:30:24.000000000 -0700 +++ devel-akpm/drivers/scsi/dc395x.c 2006-04-30 00:30:24.000000000 -0700 @@ -230,13 +230,12 @@ struct ScsiReqBlk { struct scsi_cmnd *cmd; struct SGentry *segment_x; /* Linear array of hw sg entries (up to 64 entries) */ - u32 sg_bus_addr; /* Bus address of sg list (ie, of segment_x) */ + dma_addr_t sg_bus_addr; /* Bus address of sg list (ie, of segment_x) */ u8 sg_count; /* No of HW sg entries for this request */ u8 sg_index; /* Index of HW sg entry for this request */ - u32 total_xfer_length; /* Total number of bytes remaining to be transfered */ - unsigned char *virt_addr; /* Virtual address of current transfer position */ - + size_t total_xfer_length; /* Total number of bytes remaining to be transfered */ + size_t request_length; /* Total number of bytes in this request */ /* * The sense buffer handling function, request_sense, uses * the first hw sg entry (segment_x[0]) and the transfer @@ -246,8 +245,7 @@ struct ScsiReqBlk { * total_xfer_length in xferred. These values are restored in * pci_unmap_srb_sense. This is the only place xferred is used. */ - unsigned char *virt_addr_req; /* Saved virtual address of the request buffer */ - u32 xferred; /* Saved copy of total_xfer_length */ + size_t xferred; /* Saved copy of total_xfer_length */ u16 state; @@ -977,17 +975,6 @@ static void send_srb(struct AdapterCtlBl } } -static inline void pio_trigger(void) -{ - static int feedback_requested; - - if (!feedback_requested) { - feedback_requested = 1; - printk(KERN_WARNING "%s: Please, contact " - "to help improve support for your system.\n", __FILE__); - } -} - /* Prepare SRB for being sent to Device DCB w/ command *cmd */ static void build_srb(struct scsi_cmnd *cmd, struct DeviceCtlBlk *dcb, struct ScsiReqBlk *srb) @@ -1001,7 +988,6 @@ static void build_srb(struct scsi_cmnd * srb->sg_count = 0; srb->total_xfer_length = 0; srb->sg_bus_addr = 0; - srb->virt_addr = NULL; srb->sg_index = 0; srb->adapter_status = 0; srb->target_status = 0; @@ -1032,7 +1018,6 @@ static void build_srb(struct scsi_cmnd * reqlen, cmd->request_buffer, cmd->use_sg, srb->sg_count); - srb->virt_addr = page_address(sl->page); for (i = 0; i < srb->sg_count; i++) { u32 busaddr = (u32)sg_dma_address(&sl[i]); u32 seglen = (u32)sl[i].length; @@ -1077,12 +1062,14 @@ static void build_srb(struct scsi_cmnd * srb->total_xfer_length++; srb->segment_x[0].length = srb->total_xfer_length; - srb->virt_addr = cmd->request_buffer; + dprintkdbg(DBG_0, "build_srb: [1] len=%d buf=%p use_sg=%d map=%08x\n", srb->total_xfer_length, cmd->request_buffer, cmd->use_sg, srb->segment_x[0].address); } + + srb->request_length = srb->total_xfer_length; } @@ -1414,10 +1401,10 @@ static int dc395x_eh_abort(struct scsi_c } srb = find_cmd(cmd, &dcb->srb_going_list); if (srb) { - dprintkl(KERN_DEBUG, "eh_abort: Command in progress"); + dprintkl(KERN_DEBUG, "eh_abort: Command in progress\n"); /* XXX: Should abort the command here */ } else { - dprintkl(KERN_DEBUG, "eh_abort: Command not found"); + dprintkl(KERN_DEBUG, "eh_abort: Command not found\n"); } return FAILED; } @@ -1976,14 +1963,11 @@ static void sg_verify_length(struct Scsi /* * Compute the next Scatter Gather list index and adjust its length - * and address if necessary; also compute virt_addr + * and address if necessary */ static void sg_update_list(struct ScsiReqBlk *srb, u32 left) { u8 idx; - struct scatterlist *sg; - struct scsi_cmnd *cmd = srb->cmd; - int segment = cmd->use_sg; u32 xferred = srb->total_xfer_length - left; /* bytes transfered */ struct SGentry *psge = srb->segment_x + srb->sg_index; @@ -2016,29 +2000,6 @@ static void sg_update_list(struct ScsiRe psge++; } sg_verify_length(srb); - - /* we need the corresponding virtual address */ - if (!segment || (srb->flag & AUTO_REQSENSE)) { - srb->virt_addr += xferred; - return; - } - - /* We have to walk the scatterlist to find it */ - sg = (struct scatterlist *)cmd->request_buffer; - while (segment--) { - unsigned long mask = - ~((unsigned long)sg->length - 1) & PAGE_MASK; - if ((sg_dma_address(sg) & mask) == (psge->address & mask)) { - srb->virt_addr = (page_address(sg->page) - + psge->address - - (psge->address & PAGE_MASK)); - return; - } - ++sg; - } - - dprintkl(KERN_ERR, "sg_update_list: sg_to_virt failed\n"); - srb->virt_addr = NULL; } @@ -2050,15 +2011,7 @@ static void sg_update_list(struct ScsiRe */ static void sg_subtract_one(struct ScsiReqBlk *srb) { - srb->total_xfer_length--; - srb->segment_x[srb->sg_index].length--; - if (srb->total_xfer_length && - !srb->segment_x[srb->sg_index].length) { - if (debug_enabled(DBG_PIO)) - printk(" (next segment)"); - srb->sg_index++; - sg_update_list(srb, srb->total_xfer_length); - } + sg_update_list(srb, srb->total_xfer_length - 1); } @@ -2118,7 +2071,7 @@ static void data_out_phase0(struct Adapt * If we need more data, the DMA SG list will be freshly set up, anyway */ dprintkdbg(DBG_PIO, "data_out_phase0: " - "DMA{fifcnt=0x%02x fifostat=0x%02x} " + "DMA{fifocnt=0x%02x fifostat=0x%02x} " "SCSI{fifocnt=0x%02x cnt=0x%06x status=0x%04x} total=0x%06x\n", DC395x_read8(acb, TRM_S1040_DMA_FIFOCNT), DC395x_read8(acb, TRM_S1040_DMA_FIFOSTAT), @@ -2239,12 +2192,11 @@ static void data_out_phase1(struct Adapt data_io_transfer(acb, srb, XFERDATAOUT); } - static void data_in_phase0(struct AdapterCtlBlk *acb, struct ScsiReqBlk *srb, u16 *pscsi_status) { u16 scsi_status = *pscsi_status; - u32 d_left_counter = 0; + dprintkdbg(DBG_0, "data_in_phase0: (pid#%li) <%02i-%i>\n", srb->cmd->pid, srb->cmd->device->id, srb->cmd->device->lun); @@ -2262,6 +2214,9 @@ static void data_in_phase0(struct Adapte * seem to be a bad idea, actually. */ if (!(srb->state & SRB_XFERPAD)) { + u32 d_left_counter; + unsigned int sc, fc; + if (scsi_status & PARITYERROR) { dprintkl(KERN_INFO, "data_in_phase0: (pid#%li) " "Parity Error\n", srb->cmd->pid); @@ -2298,18 +2253,19 @@ static void data_in_phase0(struct Adapte DC395x_read8(acb, TRM_S1040_DMA_FIFOSTAT)); } /* Now: Check remainig data: The SCSI counters should tell us ... */ - d_left_counter = DC395x_read32(acb, TRM_S1040_SCSI_COUNTER) - + ((DC395x_read8(acb, TRM_S1040_SCSI_FIFOCNT) & 0x1f) + sc = DC395x_read32(acb, TRM_S1040_SCSI_COUNTER); + fc = DC395x_read8(acb, TRM_S1040_SCSI_FIFOCNT); + d_left_counter = sc + ((fc & 0x1f) << ((srb->dcb->sync_period & WIDE_SYNC) ? 1 : 0)); dprintkdbg(DBG_KG, "data_in_phase0: " "SCSI{fifocnt=0x%02x%s ctr=0x%08x} " "DMA{fifocnt=0x%02x fifostat=0x%02x ctr=0x%08x} " "Remain{totxfer=%i scsi_fifo+ctr=%i}\n", - DC395x_read8(acb, TRM_S1040_SCSI_FIFOCNT), + fc, (srb->dcb->sync_period & WIDE_SYNC) ? "words" : "bytes", - DC395x_read32(acb, TRM_S1040_SCSI_COUNTER), - DC395x_read8(acb, TRM_S1040_DMA_FIFOCNT), + sc, + fc, DC395x_read8(acb, TRM_S1040_DMA_FIFOSTAT), DC395x_read32(acb, TRM_S1040_DMA_CXCNT), srb->total_xfer_length, d_left_counter); @@ -2317,40 +2273,79 @@ static void data_in_phase0(struct Adapte /* KG: Less than or equal to 4 bytes can not be transfered via DMA, it seems. */ if (d_left_counter && srb->total_xfer_length <= DC395x_LASTPIO) { + size_t left_io = srb->total_xfer_length; + /*u32 addr = (srb->segment_x[srb->sg_index].address); */ /*sg_update_list (srb, d_left_counter); */ - dprintkdbg(DBG_PIO, "data_in_phase0: PIO (%i %s) to " - "%p for remaining %i bytes:", - DC395x_read8(acb, TRM_S1040_SCSI_FIFOCNT) & 0x1f, + dprintkdbg(DBG_PIO, "data_in_phase0: PIO (%i %s) " + "for remaining %i bytes:", + fc & 0x1f, (srb->dcb->sync_period & WIDE_SYNC) ? "words" : "bytes", - srb->virt_addr, srb->total_xfer_length); if (srb->dcb->sync_period & WIDE_SYNC) DC395x_write8(acb, TRM_S1040_SCSI_CONFIG2, CFG2_WIDEFIFO); - while (DC395x_read8(acb, TRM_S1040_SCSI_FIFOCNT) != 0x40) { - u8 byte = DC395x_read8(acb, TRM_S1040_SCSI_FIFO); - pio_trigger(); - *(srb->virt_addr)++ = byte; - if (debug_enabled(DBG_PIO)) - printk(" %02x", byte); - d_left_counter--; - sg_subtract_one(srb); - } - if (srb->dcb->sync_period & WIDE_SYNC) { -#if 1 - /* Read the last byte ... */ - if (srb->total_xfer_length > 0) { - u8 byte = DC395x_read8(acb, TRM_S1040_SCSI_FIFO); - pio_trigger(); - *(srb->virt_addr)++ = byte; - srb->total_xfer_length--; + while (left_io) { + unsigned char *virt, *base = NULL; + unsigned long flags = 0; + size_t len = left_io; + + if (srb->cmd->use_sg) { + size_t offset = srb->request_length - left_io; + local_irq_save(flags); + /* Assumption: it's inside one page as it's at most 4 bytes and + I just assume it's on a 4-byte boundary */ + base = scsi_kmap_atomic_sg((struct scatterlist *)srb->cmd->request_buffer, + srb->sg_count, &offset, &len); + virt = base + offset; + } else { + virt = srb->cmd->request_buffer + srb->cmd->request_bufflen - left_io; + len = left_io; + } + left_io -= len; + + while (len) { + u8 byte; + byte = DC395x_read8(acb, TRM_S1040_SCSI_FIFO); + *virt++ = byte; + if (debug_enabled(DBG_PIO)) printk(" %02x", byte); + + d_left_counter--; + sg_subtract_one(srb); + + len--; + + fc = DC395x_read8(acb, TRM_S1040_SCSI_FIFOCNT); + + if (fc == 0x40) { + left_io = 0; + break; + } + } + + WARN_ON((fc != 0x40) == !d_left_counter); + + if (fc == 0x40 && (srb->dcb->sync_period & WIDE_SYNC)) { + /* Read the last byte ... */ + if (srb->total_xfer_length > 0) { + u8 byte = DC395x_read8(acb, TRM_S1040_SCSI_FIFO); + + *virt++ = byte; + srb->total_xfer_length--; + if (debug_enabled(DBG_PIO)) + printk(" %02x", byte); + } + + DC395x_write8(acb, TRM_S1040_SCSI_CONFIG2, 0); + } + + if (srb->cmd->use_sg) { + scsi_kunmap_atomic_sg(base); + local_irq_restore(flags); } -#endif - DC395x_write8(acb, TRM_S1040_SCSI_CONFIG2, 0); } /*printk(" %08x", *(u32*)(bus_to_virt (addr))); */ /*srb->total_xfer_length = 0; */ @@ -2509,22 +2504,43 @@ static void data_io_transfer(struct Adap SCMD_FIFO_IN); } else { /* write */ int ln = srb->total_xfer_length; + size_t left_io = srb->total_xfer_length; + if (srb->dcb->sync_period & WIDE_SYNC) DC395x_write8(acb, TRM_S1040_SCSI_CONFIG2, CFG2_WIDEFIFO); - dprintkdbg(DBG_PIO, - "data_io_transfer: PIO %i bytes from %p:", - srb->total_xfer_length, srb->virt_addr); - - while (srb->total_xfer_length) { - if (debug_enabled(DBG_PIO)) - printk(" %02x", (unsigned char) *(srb->virt_addr)); - - pio_trigger(); - DC395x_write8(acb, TRM_S1040_SCSI_FIFO, - *(srb->virt_addr)++); - sg_subtract_one(srb); + while (left_io) { + unsigned char *virt, *base = NULL; + unsigned long flags = 0; + size_t len = left_io; + + if (srb->cmd->use_sg) { + size_t offset = srb->request_length - left_io; + local_irq_save(flags); + /* Again, max 4 bytes */ + base = scsi_kmap_atomic_sg((struct scatterlist *)srb->cmd->request_buffer, + srb->sg_count, &offset, &len); + virt = base + offset; + } else { + virt = srb->cmd->request_buffer + srb->cmd->request_bufflen - left_io; + len = left_io; + } + left_io -= len; + + while (len--) { + if (debug_enabled(DBG_PIO)) + printk(" %02x", *virt); + + DC395x_write8(acb, TRM_S1040_SCSI_FIFO, *virt++); + + sg_subtract_one(srb); + } + + if (srb->cmd->use_sg) { + scsi_kunmap_atomic_sg(base); + local_irq_restore(flags); + } } if (srb->dcb->sync_period & WIDE_SYNC) { if (ln % 2) { @@ -3319,7 +3335,6 @@ static void pci_unmap_srb_sense(struct A srb->segment_x[DC395x_MAX_SG_LISTENTRY - 1].address; srb->segment_x[0].length = srb->segment_x[DC395x_MAX_SG_LISTENTRY - 1].length; - srb->virt_addr = srb->virt_addr_req; } @@ -3713,8 +3728,6 @@ static void request_sense(struct Adapter srb->xferred = srb->total_xfer_length; /* srb->segment_x : a one entry of S/G list table */ srb->total_xfer_length = sizeof(cmd->sense_buffer); - srb->virt_addr_req = srb->virt_addr; - srb->virt_addr = cmd->sense_buffer; srb->segment_x[0].length = sizeof(cmd->sense_buffer); /* Map sense buffer */ srb->segment_x[0].address = diff -puN drivers/scsi/ibmvscsi/ibmvscsi.c~git-scsi-misc drivers/scsi/ibmvscsi/ibmvscsi.c --- devel/drivers/scsi/ibmvscsi/ibmvscsi.c~git-scsi-misc 2006-04-30 00:30:24.000000000 -0700 +++ devel-akpm/drivers/scsi/ibmvscsi/ibmvscsi.c 2006-04-30 00:30:24.000000000 -0700 @@ -121,10 +121,9 @@ static int initialize_event_pool(struct pool->size = size; pool->next = 0; - pool->events = kmalloc(pool->size * sizeof(*pool->events), GFP_KERNEL); + pool->events = kcalloc(pool->size, sizeof(*pool->events), GFP_KERNEL); if (!pool->events) return -ENOMEM; - memset(pool->events, 0x00, pool->size * sizeof(*pool->events)); pool->iu_storage = dma_alloc_coherent(hostdata->dev, diff -puN drivers/scsi/iscsi_tcp.c~git-scsi-misc drivers/scsi/iscsi_tcp.c --- devel/drivers/scsi/iscsi_tcp.c~git-scsi-misc 2006-04-30 00:30:24.000000000 -0700 +++ devel-akpm/drivers/scsi/iscsi_tcp.c 2006-04-30 00:30:24.000000000 -0700 @@ -3,7 +3,8 @@ * * Copyright (C) 2004 Dmitry Yusupov * Copyright (C) 2004 Alex Aizman - * Copyright (C) 2005 Mike Christie + * Copyright (C) 2005 - 2006 Mike Christie + * Copyright (C) 2006 Red Hat, Inc. All rights reserved. * maintained by open-iscsi@googlegroups.com * * This program is free software; you can redistribute it and/or modify @@ -36,10 +37,6 @@ #include #include #include -#include -#include -#include -#include #include #include #include @@ -52,21 +49,14 @@ MODULE_DESCRIPTION("iSCSI/TCP data-path" MODULE_LICENSE("GPL"); MODULE_VERSION("0:4.445"); /* #define DEBUG_TCP */ -/* #define DEBUG_SCSI */ #define DEBUG_ASSERT #ifdef DEBUG_TCP -#define debug_tcp(fmt...) printk(KERN_DEBUG "tcp: " fmt) +#define debug_tcp(fmt...) printk(KERN_INFO "tcp: " fmt) #else #define debug_tcp(fmt...) #endif -#ifdef DEBUG_SCSI -#define debug_scsi(fmt...) printk(KERN_DEBUG "scsi: " fmt) -#else -#define debug_scsi(fmt...) -#endif - #ifndef DEBUG_ASSERT #ifdef BUG_ON #undef BUG_ON @@ -74,8 +64,6 @@ MODULE_VERSION("0:4.445"); #define BUG_ON(expr) #endif -#define INVALID_SN_DELTA 0xffff - static unsigned int iscsi_max_lun = 512; module_param_named(max_lun, iscsi_max_lun, uint, S_IRUGO); @@ -130,68 +118,39 @@ static inline void iscsi_hdr_digest(struct iscsi_conn *conn, struct iscsi_buf *buf, u8* crc) { - crypto_digest_digest(conn->tx_tfm, &buf->sg, 1, crc); - buf->sg.length += sizeof(uint32_t); -} - -static void -iscsi_conn_failure(struct iscsi_conn *conn, enum iscsi_err err) -{ - struct iscsi_session *session = conn->session; - unsigned long flags; + struct iscsi_tcp_conn *tcp_conn = conn->dd_data; - spin_lock_irqsave(&session->lock, flags); - if (session->conn_cnt == 1 || session->leadconn == conn) - session->state = ISCSI_STATE_FAILED; - spin_unlock_irqrestore(&session->lock, flags); - set_bit(SUSPEND_BIT, &conn->suspend_tx); - set_bit(SUSPEND_BIT, &conn->suspend_rx); - iscsi_conn_error(conn->cls_conn, err); + crypto_digest_digest(tcp_conn->tx_tfm, &buf->sg, 1, crc); + buf->sg.length += sizeof(uint32_t); } static inline int -iscsi_check_assign_cmdsn(struct iscsi_session *session, struct iscsi_nopin *hdr) +iscsi_hdr_extract(struct iscsi_tcp_conn *tcp_conn) { - uint32_t max_cmdsn = be32_to_cpu(hdr->max_cmdsn); - uint32_t exp_cmdsn = be32_to_cpu(hdr->exp_cmdsn); - - if (max_cmdsn < exp_cmdsn -1 && - max_cmdsn > exp_cmdsn - INVALID_SN_DELTA) - return ISCSI_ERR_MAX_CMDSN; - if (max_cmdsn > session->max_cmdsn || - max_cmdsn < session->max_cmdsn - INVALID_SN_DELTA) - session->max_cmdsn = max_cmdsn; - if (exp_cmdsn > session->exp_cmdsn || - exp_cmdsn < session->exp_cmdsn - INVALID_SN_DELTA) - session->exp_cmdsn = exp_cmdsn; + struct sk_buff *skb = tcp_conn->in.skb; - return 0; -} - -static inline int -iscsi_hdr_extract(struct iscsi_conn *conn) -{ - struct sk_buff *skb = conn->in.skb; + tcp_conn->in.zero_copy_hdr = 0; - if (conn->in.copy >= conn->hdr_size && - conn->in_progress == IN_PROGRESS_WAIT_HEADER) { + if (tcp_conn->in.copy >= tcp_conn->hdr_size && + tcp_conn->in_progress == IN_PROGRESS_WAIT_HEADER) { /* * Zero-copy PDU Header: using connection context * to store header pointer. */ if (skb_shinfo(skb)->frag_list == NULL && - !skb_shinfo(skb)->nr_frags) - conn->in.hdr = (struct iscsi_hdr *) - ((char*)skb->data + conn->in.offset); - else { + !skb_shinfo(skb)->nr_frags) { + tcp_conn->in.hdr = (struct iscsi_hdr *) + ((char*)skb->data + tcp_conn->in.offset); + tcp_conn->in.zero_copy_hdr = 1; + } else { /* ignoring return code since we checked * in.copy before */ - skb_copy_bits(skb, conn->in.offset, - &conn->hdr, conn->hdr_size); - conn->in.hdr = &conn->hdr; + skb_copy_bits(skb, tcp_conn->in.offset, + &tcp_conn->hdr, tcp_conn->hdr_size); + tcp_conn->in.hdr = &tcp_conn->hdr; } - conn->in.offset += conn->hdr_size; - conn->in.copy -= conn->hdr_size; + tcp_conn->in.offset += tcp_conn->hdr_size; + tcp_conn->in.copy -= tcp_conn->hdr_size; } else { int hdr_remains; int copylen; @@ -201,118 +160,61 @@ iscsi_hdr_extract(struct iscsi_conn *con * copying it... This'll happen quite rarely. */ - if (conn->in_progress == IN_PROGRESS_WAIT_HEADER) - conn->in.hdr_offset = 0; + if (tcp_conn->in_progress == IN_PROGRESS_WAIT_HEADER) + tcp_conn->in.hdr_offset = 0; - hdr_remains = conn->hdr_size - conn->in.hdr_offset; + hdr_remains = tcp_conn->hdr_size - tcp_conn->in.hdr_offset; BUG_ON(hdr_remains <= 0); - copylen = min(conn->in.copy, hdr_remains); - skb_copy_bits(skb, conn->in.offset, - (char*)&conn->hdr + conn->in.hdr_offset, copylen); + copylen = min(tcp_conn->in.copy, hdr_remains); + skb_copy_bits(skb, tcp_conn->in.offset, + (char*)&tcp_conn->hdr + tcp_conn->in.hdr_offset, + copylen); debug_tcp("PDU gather offset %d bytes %d in.offset %d " - "in.copy %d\n", conn->in.hdr_offset, copylen, - conn->in.offset, conn->in.copy); + "in.copy %d\n", tcp_conn->in.hdr_offset, copylen, + tcp_conn->in.offset, tcp_conn->in.copy); - conn->in.offset += copylen; - conn->in.copy -= copylen; + tcp_conn->in.offset += copylen; + tcp_conn->in.copy -= copylen; if (copylen < hdr_remains) { - conn->in_progress = IN_PROGRESS_HEADER_GATHER; - conn->in.hdr_offset += copylen; + tcp_conn->in_progress = IN_PROGRESS_HEADER_GATHER; + tcp_conn->in.hdr_offset += copylen; return -EAGAIN; } - conn->in.hdr = &conn->hdr; - conn->discontiguous_hdr_cnt++; - conn->in_progress = IN_PROGRESS_WAIT_HEADER; + tcp_conn->in.hdr = &tcp_conn->hdr; + tcp_conn->discontiguous_hdr_cnt++; + tcp_conn->in_progress = IN_PROGRESS_WAIT_HEADER; } return 0; } -static inline void -iscsi_ctask_cleanup(struct iscsi_conn *conn, struct iscsi_cmd_task *ctask) +/* + * must be called with session lock + */ +static void +__iscsi_ctask_cleanup(struct iscsi_conn *conn, struct iscsi_cmd_task *ctask) { - struct scsi_cmnd *sc = ctask->sc; - struct iscsi_session *session = conn->session; + struct iscsi_tcp_cmd_task *tcp_ctask = ctask->dd_data; + struct scsi_cmnd *sc; - spin_lock(&session->lock); - if (unlikely(!sc)) { - spin_unlock(&session->lock); + sc = ctask->sc; + if (unlikely(!sc)) return; - } + if (sc->sc_data_direction == DMA_TO_DEVICE) { struct iscsi_data_task *dtask, *n; + /* WRITE: cleanup Data-Out's if any */ - list_for_each_entry_safe(dtask, n, &ctask->dataqueue, item) { + list_for_each_entry_safe(dtask, n, &tcp_ctask->dataqueue, + item) { list_del(&dtask->item); - mempool_free(dtask, ctask->datapool); + mempool_free(dtask, tcp_ctask->datapool); } } - ctask->xmstate = XMSTATE_IDLE; - ctask->r2t = NULL; - ctask->sc = NULL; - __kfifo_put(session->cmdpool.queue, (void*)&ctask, sizeof(void*)); - spin_unlock(&session->lock); -} - -/** - * iscsi_cmd_rsp - SCSI Command Response processing - * @conn: iscsi connection - * @ctask: scsi command task - **/ -static int -iscsi_cmd_rsp(struct iscsi_conn *conn, struct iscsi_cmd_task *ctask) -{ - int rc; - struct iscsi_cmd_rsp *rhdr = (struct iscsi_cmd_rsp *)conn->in.hdr; - struct iscsi_session *session = conn->session; - struct scsi_cmnd *sc = ctask->sc; - - rc = iscsi_check_assign_cmdsn(session, (struct iscsi_nopin*)rhdr); - if (rc) { - sc->result = (DID_ERROR << 16); - goto out; - } - - conn->exp_statsn = be32_to_cpu(rhdr->statsn) + 1; - - sc->result = (DID_OK << 16) | rhdr->cmd_status; - - if (rhdr->response != ISCSI_STATUS_CMD_COMPLETED) { - sc->result = (DID_ERROR << 16); - goto out; - } - - if (rhdr->cmd_status == SAM_STAT_CHECK_CONDITION && conn->senselen) { - int sensecopy = min(conn->senselen, SCSI_SENSE_BUFFERSIZE); - - memcpy(sc->sense_buffer, conn->data + 2, sensecopy); - debug_scsi("copied %d bytes of sense\n", sensecopy); - } - - if (sc->sc_data_direction == DMA_TO_DEVICE) - goto out; - - if (rhdr->flags & ISCSI_FLAG_CMD_UNDERFLOW) { - int res_count = be32_to_cpu(rhdr->residual_count); - - if (res_count > 0 && res_count <= sc->request_bufflen) - sc->resid = res_count; - else - sc->result = (DID_BAD_TARGET << 16) | rhdr->cmd_status; - } else if (rhdr->flags & ISCSI_FLAG_CMD_BIDI_UNDERFLOW) - sc->result = (DID_BAD_TARGET << 16) | rhdr->cmd_status; - else if (rhdr->flags & ISCSI_FLAG_CMD_OVERFLOW) - sc->resid = be32_to_cpu(rhdr->residual_count); - -out: - debug_scsi("done [sc %lx res %d itt 0x%x]\n", - (long)sc, sc->result, ctask->itt); - conn->scsirsp_pdus_cnt++; - iscsi_ctask_cleanup(conn, ctask); - sc->scsi_done(sc); - return rc; + tcp_ctask->xmstate = XMSTATE_IDLE; + tcp_ctask->r2t = NULL; } /** @@ -324,7 +226,9 @@ static int iscsi_data_rsp(struct iscsi_conn *conn, struct iscsi_cmd_task *ctask) { int rc; - struct iscsi_data_rsp *rhdr = (struct iscsi_data_rsp *)conn->in.hdr; + struct iscsi_tcp_conn *tcp_conn = conn->dd_data; + struct iscsi_tcp_cmd_task *tcp_ctask = ctask->dd_data; + struct iscsi_data_rsp *rhdr = (struct iscsi_data_rsp *)tcp_conn->in.hdr; struct iscsi_session *session = conn->session; int datasn = be32_to_cpu(rhdr->datasn); @@ -334,9 +238,9 @@ iscsi_data_rsp(struct iscsi_conn *conn, /* * setup Data-In byte counter (gets decremented..) */ - ctask->data_count = conn->in.datalen; + ctask->data_count = tcp_conn->in.datalen; - if (conn->in.datalen == 0) + if (tcp_conn->in.datalen == 0) return 0; if (ctask->datasn != datasn) @@ -344,8 +248,8 @@ iscsi_data_rsp(struct iscsi_conn *conn, ctask->datasn++; - ctask->data_offset = be32_to_cpu(rhdr->offset); - if (ctask->data_offset + conn->in.datalen > ctask->total_length) + tcp_ctask->data_offset = be32_to_cpu(rhdr->offset); + if (tcp_ctask->data_offset + tcp_conn->in.datalen > ctask->total_length) return ISCSI_ERR_DATA_OFFSET; if (rhdr->flags & ISCSI_FLAG_DATA_STATUS) { @@ -392,17 +296,19 @@ iscsi_solicit_data_init(struct iscsi_con struct iscsi_data *hdr; struct iscsi_data_task *dtask; struct scsi_cmnd *sc = ctask->sc; + struct iscsi_tcp_cmd_task *tcp_ctask = ctask->dd_data; - dtask = mempool_alloc(ctask->datapool, GFP_ATOMIC); + dtask = mempool_alloc(tcp_ctask->datapool, GFP_ATOMIC); BUG_ON(!dtask); + INIT_LIST_HEAD(&dtask->item); hdr = &dtask->hdr; memset(hdr, 0, sizeof(struct iscsi_data)); hdr->ttt = r2t->ttt; hdr->datasn = cpu_to_be32(r2t->solicit_datasn); r2t->solicit_datasn++; hdr->opcode = ISCSI_OP_SCSI_DATA_OUT; - memcpy(hdr->lun, ctask->hdr.lun, sizeof(hdr->lun)); - hdr->itt = ctask->hdr.itt; + memcpy(hdr->lun, ctask->hdr->lun, sizeof(hdr->lun)); + hdr->itt = ctask->hdr->itt; hdr->exp_statsn = r2t->exp_statsn; hdr->offset = cpu_to_be32(r2t->data_offset); if (r2t->data_length > conn->max_xmit_dlength) { @@ -451,11 +357,11 @@ iscsi_solicit_data_init(struct iscsi_con } BUG_ON(r2t->sg == NULL); } else - iscsi_buf_init_iov(&ctask->sendbuf, + iscsi_buf_init_iov(&tcp_ctask->sendbuf, (char*)sc->request_buffer + r2t->data_offset, r2t->data_count); - list_add(&dtask->item, &ctask->dataqueue); + list_add(&dtask->item, &tcp_ctask->dataqueue); } /** @@ -468,17 +374,16 @@ iscsi_r2t_rsp(struct iscsi_conn *conn, s { struct iscsi_r2t_info *r2t; struct iscsi_session *session = conn->session; - struct iscsi_r2t_rsp *rhdr = (struct iscsi_r2t_rsp *)conn->in.hdr; + struct iscsi_tcp_cmd_task *tcp_ctask = ctask->dd_data; + struct iscsi_tcp_conn *tcp_conn = conn->dd_data; + struct iscsi_r2t_rsp *rhdr = (struct iscsi_r2t_rsp *)tcp_conn->in.hdr; int r2tsn = be32_to_cpu(rhdr->r2tsn); int rc; - if (conn->in.ahslen) - return ISCSI_ERR_AHSLEN; - - if (conn->in.datalen) + if (tcp_conn->in.datalen) return ISCSI_ERR_DATALEN; - if (ctask->exp_r2tsn && ctask->exp_r2tsn != r2tsn) + if (tcp_ctask->exp_r2tsn && tcp_ctask->exp_r2tsn != r2tsn) return ISCSI_ERR_R2TSN; rc = iscsi_check_assign_cmdsn(session, (struct iscsi_nopin*)rhdr); @@ -496,7 +401,7 @@ iscsi_r2t_rsp(struct iscsi_conn *conn, s spin_unlock(&session->lock); return 0; } - rc = __kfifo_get(ctask->r2tpool.queue, (void*)&r2t, sizeof(void*)); + rc = __kfifo_get(tcp_ctask->r2tpool.queue, (void*)&r2t, sizeof(void*)); BUG_ON(!rc); r2t->exp_statsn = rhdr->statsn; @@ -518,10 +423,10 @@ iscsi_r2t_rsp(struct iscsi_conn *conn, s iscsi_solicit_data_init(conn, ctask, r2t); - ctask->exp_r2tsn = r2tsn + 1; - ctask->xmstate |= XMSTATE_SOL_HDR; - __kfifo_put(ctask->r2tqueue, (void*)&r2t, sizeof(void*)); - __kfifo_put(conn->writequeue, (void*)&ctask, sizeof(void*)); + tcp_ctask->exp_r2tsn = r2tsn + 1; + tcp_ctask->xmstate |= XMSTATE_SOL_HDR; + __kfifo_put(tcp_ctask->r2tqueue, (void*)&r2t, sizeof(void*)); + __kfifo_put(conn->xmitqueue, (void*)&ctask, sizeof(void*)); scsi_queue_work(session->host, &conn->xmitwork); conn->r2t_pdus_cnt++; @@ -531,258 +436,136 @@ iscsi_r2t_rsp(struct iscsi_conn *conn, s } static int -iscsi_hdr_recv(struct iscsi_conn *conn) +iscsi_tcp_hdr_recv(struct iscsi_conn *conn) { - int rc = 0; + int rc = 0, opcode, ahslen; struct iscsi_hdr *hdr; - struct iscsi_cmd_task *ctask; struct iscsi_session *session = conn->session; - uint32_t cdgst, rdgst = 0; + struct iscsi_tcp_conn *tcp_conn = conn->dd_data; + uint32_t cdgst, rdgst = 0, itt; - hdr = conn->in.hdr; + hdr = tcp_conn->in.hdr; /* verify PDU length */ - conn->in.datalen = ntoh24(hdr->dlength); - if (conn->in.datalen > conn->max_recv_dlength) { + tcp_conn->in.datalen = ntoh24(hdr->dlength); + if (tcp_conn->in.datalen > conn->max_recv_dlength) { printk(KERN_ERR "iscsi_tcp: datalen %d > %d\n", - conn->in.datalen, conn->max_recv_dlength); + tcp_conn->in.datalen, conn->max_recv_dlength); return ISCSI_ERR_DATALEN; } - conn->data_copied = 0; + tcp_conn->data_copied = 0; /* read AHS */ - conn->in.ahslen = hdr->hlength * 4; - conn->in.offset += conn->in.ahslen; - conn->in.copy -= conn->in.ahslen; - if (conn->in.copy < 0) { + ahslen = hdr->hlength << 2; + tcp_conn->in.offset += ahslen; + tcp_conn->in.copy -= ahslen; + if (tcp_conn->in.copy < 0) { printk(KERN_ERR "iscsi_tcp: can't handle AHS with length " - "%d bytes\n", conn->in.ahslen); + "%d bytes\n", ahslen); return ISCSI_ERR_AHSLEN; } /* calculate read padding */ - conn->in.padding = conn->in.datalen & (ISCSI_PAD_LEN-1); - if (conn->in.padding) { - conn->in.padding = ISCSI_PAD_LEN - conn->in.padding; - debug_scsi("read padding %d bytes\n", conn->in.padding); + tcp_conn->in.padding = tcp_conn->in.datalen & (ISCSI_PAD_LEN-1); + if (tcp_conn->in.padding) { + tcp_conn->in.padding = ISCSI_PAD_LEN - tcp_conn->in.padding; + debug_scsi("read padding %d bytes\n", tcp_conn->in.padding); } if (conn->hdrdgst_en) { struct scatterlist sg; sg_init_one(&sg, (u8 *)hdr, - sizeof(struct iscsi_hdr) + conn->in.ahslen); - crypto_digest_digest(conn->rx_tfm, &sg, 1, (u8 *)&cdgst); + sizeof(struct iscsi_hdr) + ahslen); + crypto_digest_digest(tcp_conn->rx_tfm, &sg, 1, (u8 *)&cdgst); rdgst = *(uint32_t*)((char*)hdr + sizeof(struct iscsi_hdr) + - conn->in.ahslen); + ahslen); if (cdgst != rdgst) { - printk(KERN_ERR "iscsi_tcp: itt %x: hdrdgst error " - "recv 0x%x calc 0x%x\n", conn->in.itt, rdgst, - cdgst); + printk(KERN_ERR "iscsi_tcp: hdrdgst error " + "recv 0x%x calc 0x%x\n", rdgst, cdgst); return ISCSI_ERR_HDR_DGST; } } - /* save opcode for later */ - conn->in.opcode = hdr->opcode & ISCSI_OPCODE_MASK; - + opcode = hdr->opcode & ISCSI_OPCODE_MASK; /* verify itt (itt encoding: age+cid+itt) */ - if (hdr->itt != cpu_to_be32(ISCSI_RESERVED_TAG)) { - if ((hdr->itt & AGE_MASK) != - (session->age << AGE_SHIFT)) { - printk(KERN_ERR "iscsi_tcp: received itt %x expected " - "session age (%x)\n", hdr->itt, - session->age & AGE_MASK); - return ISCSI_ERR_BAD_ITT; - } - - if ((hdr->itt & CID_MASK) != (conn->id << CID_SHIFT)) { - printk(KERN_ERR "iscsi_tcp: received itt %x, expected " - "CID (%x)\n", hdr->itt, conn->id); - return ISCSI_ERR_BAD_ITT; - } - conn->in.itt = hdr->itt & ITT_MASK; - } else - conn->in.itt = hdr->itt; + rc = iscsi_verify_itt(conn, hdr, &itt); + if (rc == ISCSI_ERR_NO_SCSI_CMD) { + tcp_conn->in.datalen = 0; /* force drop */ + return 0; + } else if (rc) + return rc; debug_tcp("opcode 0x%x offset %d copy %d ahslen %d datalen %d\n", - hdr->opcode, conn->in.offset, conn->in.copy, - conn->in.ahslen, conn->in.datalen); - - if (conn->in.itt < session->cmds_max) { - ctask = (struct iscsi_cmd_task *)session->cmds[conn->in.itt]; - - if (!ctask->sc) { - printk(KERN_INFO "iscsi_tcp: dropping ctask with " - "itt 0x%x\n", ctask->itt); - conn->in.datalen = 0; /* force drop */ - return 0; - } - - if (ctask->sc->SCp.phase != session->age) { - printk(KERN_ERR "iscsi_tcp: ctask's session age %d, " - "expected %d\n", ctask->sc->SCp.phase, - session->age); - return ISCSI_ERR_SESSION_FAILED; - } - - conn->in.ctask = ctask; - - debug_scsi("rsp [op 0x%x cid %d sc %lx itt 0x%x len %d]\n", - hdr->opcode, conn->id, (long)ctask->sc, - ctask->itt, conn->in.datalen); - - switch(conn->in.opcode) { - case ISCSI_OP_SCSI_CMD_RSP: - BUG_ON((void*)ctask != ctask->sc->SCp.ptr); - if (!conn->in.datalen) - rc = iscsi_cmd_rsp(conn, ctask); - else - /* - * got sense or response data; copying PDU - * Header to the connection's header - * placeholder - */ - memcpy(&conn->hdr, hdr, - sizeof(struct iscsi_hdr)); - break; - case ISCSI_OP_SCSI_DATA_IN: - BUG_ON((void*)ctask != ctask->sc->SCp.ptr); - /* save flags for non-exceptional status */ - conn->in.flags = hdr->flags; - /* save cmd_status for sense data */ - conn->in.cmd_status = - ((struct iscsi_data_rsp*)hdr)->cmd_status; - rc = iscsi_data_rsp(conn, ctask); - break; - case ISCSI_OP_R2T: - BUG_ON((void*)ctask != ctask->sc->SCp.ptr); - if (ctask->sc->sc_data_direction == DMA_TO_DEVICE) - rc = iscsi_r2t_rsp(conn, ctask); - else - rc = ISCSI_ERR_PROTO; - break; - default: - rc = ISCSI_ERR_BAD_OPCODE; - break; - } - } else if (conn->in.itt >= ISCSI_MGMT_ITT_OFFSET && - conn->in.itt < ISCSI_MGMT_ITT_OFFSET + - session->mgmtpool_max) { - struct iscsi_mgmt_task *mtask = (struct iscsi_mgmt_task *) - session->mgmt_cmds[conn->in.itt - - ISCSI_MGMT_ITT_OFFSET]; - - debug_scsi("immrsp [op 0x%x cid %d itt 0x%x len %d]\n", - conn->in.opcode, conn->id, mtask->itt, - conn->in.datalen); - - switch(conn->in.opcode) { - case ISCSI_OP_LOGIN_RSP: - case ISCSI_OP_TEXT_RSP: - case ISCSI_OP_LOGOUT_RSP: - rc = iscsi_check_assign_cmdsn(session, - (struct iscsi_nopin*)hdr); - if (rc) - break; + opcode, tcp_conn->in.offset, tcp_conn->in.copy, + ahslen, tcp_conn->in.datalen); - if (!conn->in.datalen) { - rc = iscsi_recv_pdu(conn->cls_conn, hdr, - NULL, 0); - if (conn->login_mtask != mtask) { - spin_lock(&session->lock); - __kfifo_put(session->mgmtpool.queue, - (void*)&mtask, sizeof(void*)); - spin_unlock(&session->lock); - } - } - break; - case ISCSI_OP_SCSI_TMFUNC_RSP: - rc = iscsi_check_assign_cmdsn(session, - (struct iscsi_nopin*)hdr); - if (rc) - break; - - if (conn->in.datalen || conn->in.ahslen) { - rc = ISCSI_ERR_PROTO; - break; - } - conn->tmfrsp_pdus_cnt++; - spin_lock(&session->lock); - if (conn->tmabort_state == TMABORT_INITIAL) { - __kfifo_put(session->mgmtpool.queue, - (void*)&mtask, sizeof(void*)); - conn->tmabort_state = - ((struct iscsi_tm_rsp *)hdr)-> - response == ISCSI_TMF_RSP_COMPLETE ? - TMABORT_SUCCESS:TMABORT_FAILED; - /* unblock eh_abort() */ - wake_up(&conn->ehwait); - } - spin_unlock(&session->lock); - break; - case ISCSI_OP_NOOP_IN: - if (hdr->ttt != ISCSI_RESERVED_TAG) { - rc = ISCSI_ERR_PROTO; - break; - } - rc = iscsi_check_assign_cmdsn(session, - (struct iscsi_nopin*)hdr); - if (rc) - break; - conn->exp_statsn = be32_to_cpu(hdr->statsn) + 1; - - if (!conn->in.datalen) { - struct iscsi_mgmt_task *mtask; - - rc = iscsi_recv_pdu(conn->cls_conn, hdr, - NULL, 0); - mtask = (struct iscsi_mgmt_task *) - session->mgmt_cmds[conn->in.itt - - ISCSI_MGMT_ITT_OFFSET]; - if (conn->login_mtask != mtask) { - spin_lock(&session->lock); - __kfifo_put(session->mgmtpool.queue, - (void*)&mtask, sizeof(void*)); - spin_unlock(&session->lock); - } - } - break; - default: - rc = ISCSI_ERR_BAD_OPCODE; - break; - } - } else if (conn->in.itt == ISCSI_RESERVED_TAG) { - switch(conn->in.opcode) { - case ISCSI_OP_NOOP_IN: - if (!conn->in.datalen) { - rc = iscsi_check_assign_cmdsn(session, - (struct iscsi_nopin*)hdr); - if (!rc && hdr->ttt != ISCSI_RESERVED_TAG) - rc = iscsi_recv_pdu(conn->cls_conn, - hdr, NULL, 0); - } else - rc = ISCSI_ERR_PROTO; - break; - case ISCSI_OP_REJECT: - /* we need sth like iscsi_reject_rsp()*/ - case ISCSI_OP_ASYNC_EVENT: - /* we need sth like iscsi_async_event_rsp() */ - rc = ISCSI_ERR_BAD_OPCODE; - break; - default: - rc = ISCSI_ERR_BAD_OPCODE; - break; - } - } else - rc = ISCSI_ERR_BAD_ITT; + switch(opcode) { + case ISCSI_OP_SCSI_DATA_IN: + tcp_conn->in.ctask = session->cmds[itt]; + rc = iscsi_data_rsp(conn, tcp_conn->in.ctask); + /* fall through */ + case ISCSI_OP_SCSI_CMD_RSP: + tcp_conn->in.ctask = session->cmds[itt]; + if (tcp_conn->in.datalen) + goto copy_hdr; + + spin_lock(&session->lock); + __iscsi_ctask_cleanup(conn, tcp_conn->in.ctask); + rc = __iscsi_complete_pdu(conn, hdr, NULL, 0); + spin_unlock(&session->lock); + break; + case ISCSI_OP_R2T: + tcp_conn->in.ctask = session->cmds[itt]; + if (ahslen) + rc = ISCSI_ERR_AHSLEN; + else if (tcp_conn->in.ctask->sc->sc_data_direction == + DMA_TO_DEVICE) + rc = iscsi_r2t_rsp(conn, tcp_conn->in.ctask); + else + rc = ISCSI_ERR_PROTO; + break; + case ISCSI_OP_LOGIN_RSP: + case ISCSI_OP_TEXT_RSP: + case ISCSI_OP_LOGOUT_RSP: + case ISCSI_OP_NOOP_IN: + case ISCSI_OP_REJECT: + case ISCSI_OP_ASYNC_EVENT: + if (tcp_conn->in.datalen) + goto copy_hdr; + /* fall through */ + case ISCSI_OP_SCSI_TMFUNC_RSP: + rc = iscsi_complete_pdu(conn, hdr, NULL, 0); + break; + default: + rc = ISCSI_ERR_BAD_OPCODE; + break; + } return rc; + +copy_hdr: + /* + * if we did zero copy for the header but we will need multiple + * skbs to complete the command then we have to copy the header + * for later use + */ + if (tcp_conn->in.zero_copy_hdr && tcp_conn->in.copy < + (tcp_conn->in.datalen + tcp_conn->in.padding + + (conn->datadgst_en ? 4 : 0))) { + debug_tcp("Copying header for later use. in.copy %d in.datalen" + " %d\n", tcp_conn->in.copy, tcp_conn->in.datalen); + memcpy(&tcp_conn->hdr, tcp_conn->in.hdr, + sizeof(struct iscsi_hdr)); + tcp_conn->in.hdr = &tcp_conn->hdr; + tcp_conn->in.zero_copy_hdr = 0; + } + return 0; } /** * iscsi_ctask_copy - copy skb bits to the destanation cmd task - * @conn: iscsi connection + * @conn: iscsi tcp connection * @ctask: scsi command task * @buf: buffer to copy to * @buf_size: size of buffer @@ -804,110 +587,113 @@ iscsi_hdr_recv(struct iscsi_conn *conn) * buf_left left to copy from in progress buffer **/ static inline int -iscsi_ctask_copy(struct iscsi_conn *conn, struct iscsi_cmd_task *ctask, +iscsi_ctask_copy(struct iscsi_tcp_conn *tcp_conn, struct iscsi_cmd_task *ctask, void *buf, int buf_size, int offset) { - int buf_left = buf_size - (conn->data_copied + offset); - int size = min(conn->in.copy, buf_left); + struct iscsi_tcp_cmd_task *tcp_ctask = ctask->dd_data; + int buf_left = buf_size - (tcp_conn->data_copied + offset); + int size = min(tcp_conn->in.copy, buf_left); int rc; size = min(size, ctask->data_count); debug_tcp("ctask_copy %d bytes at offset %d copied %d\n", - size, conn->in.offset, conn->in.copied); + size, tcp_conn->in.offset, tcp_conn->in.copied); BUG_ON(size <= 0); - BUG_ON(ctask->sent + size > ctask->total_length); + BUG_ON(tcp_ctask->sent + size > ctask->total_length); - rc = skb_copy_bits(conn->in.skb, conn->in.offset, - (char*)buf + (offset + conn->data_copied), size); + rc = skb_copy_bits(tcp_conn->in.skb, tcp_conn->in.offset, + (char*)buf + (offset + tcp_conn->data_copied), size); /* must fit into skb->len */ BUG_ON(rc); - conn->in.offset += size; - conn->in.copy -= size; - conn->in.copied += size; - conn->data_copied += size; - ctask->sent += size; + tcp_conn->in.offset += size; + tcp_conn->in.copy -= size; + tcp_conn->in.copied += size; + tcp_conn->data_copied += size; + tcp_ctask->sent += size; ctask->data_count -= size; - BUG_ON(conn->in.copy < 0); + BUG_ON(tcp_conn->in.copy < 0); BUG_ON(ctask->data_count < 0); - if (buf_size != (conn->data_copied + offset)) { + if (buf_size != (tcp_conn->data_copied + offset)) { if (!ctask->data_count) { - BUG_ON(buf_size - conn->data_copied < 0); + BUG_ON(buf_size - tcp_conn->data_copied < 0); /* done with this PDU */ - return buf_size - conn->data_copied; + return buf_size - tcp_conn->data_copied; } return -EAGAIN; } /* done with this buffer or with both - PDU and buffer */ - conn->data_copied = 0; + tcp_conn->data_copied = 0; return 0; } /** * iscsi_tcp_copy - copy skb bits to the destanation buffer - * @conn: iscsi connection - * @buf: buffer to copy to - * @buf_size: number of bytes to copy + * @conn: iscsi tcp connection * * Notes: * The function calls skb_copy_bits() and updates per-connection * byte counters. **/ static inline int -iscsi_tcp_copy(struct iscsi_conn *conn, void *buf, int buf_size) +iscsi_tcp_copy(struct iscsi_tcp_conn *tcp_conn) { - int buf_left = buf_size - conn->data_copied; - int size = min(conn->in.copy, buf_left); + void *buf = tcp_conn->data; + int buf_size = tcp_conn->in.datalen; + int buf_left = buf_size - tcp_conn->data_copied; + int size = min(tcp_conn->in.copy, buf_left); int rc; debug_tcp("tcp_copy %d bytes at offset %d copied %d\n", - size, conn->in.offset, conn->data_copied); + size, tcp_conn->in.offset, tcp_conn->data_copied); BUG_ON(size <= 0); - rc = skb_copy_bits(conn->in.skb, conn->in.offset, - (char*)buf + conn->data_copied, size); + rc = skb_copy_bits(tcp_conn->in.skb, tcp_conn->in.offset, + (char*)buf + tcp_conn->data_copied, size); BUG_ON(rc); - conn->in.offset += size; - conn->in.copy -= size; - conn->in.copied += size; - conn->data_copied += size; + tcp_conn->in.offset += size; + tcp_conn->in.copy -= size; + tcp_conn->in.copied += size; + tcp_conn->data_copied += size; - if (buf_size != conn->data_copied) + if (buf_size != tcp_conn->data_copied) return -EAGAIN; return 0; } static inline void -partial_sg_digest_update(struct iscsi_conn *conn, struct scatterlist *sg, - int offset, int length) +partial_sg_digest_update(struct iscsi_tcp_conn *tcp_conn, + struct scatterlist *sg, int offset, int length) { struct scatterlist temp; memcpy(&temp, sg, sizeof(struct scatterlist)); temp.offset = offset; temp.length = length; - crypto_digest_update(conn->data_rx_tfm, &temp, 1); + crypto_digest_update(tcp_conn->data_rx_tfm, &temp, 1); } static void -iscsi_recv_digest_update(struct iscsi_conn *conn, char* buf, int len) +iscsi_recv_digest_update(struct iscsi_tcp_conn *tcp_conn, char* buf, int len) { struct scatterlist tmp; sg_init_one(&tmp, buf, len); - crypto_digest_update(conn->data_rx_tfm, &tmp, 1); + crypto_digest_update(tcp_conn->data_rx_tfm, &tmp, 1); } static int iscsi_scsi_data_in(struct iscsi_conn *conn) { - struct iscsi_cmd_task *ctask = conn->in.ctask; + struct iscsi_tcp_conn *tcp_conn = conn->dd_data; + struct iscsi_cmd_task *ctask = tcp_conn->in.ctask; + struct iscsi_tcp_cmd_task *tcp_ctask = ctask->dd_data; struct scsi_cmnd *sc = ctask->sc; struct scatterlist *sg; int i, offset, rc = 0; @@ -919,31 +705,33 @@ static int iscsi_scsi_data_in(struct isc */ if (!sc->use_sg) { i = ctask->data_count; - rc = iscsi_ctask_copy(conn, ctask, sc->request_buffer, - sc->request_bufflen, ctask->data_offset); + rc = iscsi_ctask_copy(tcp_conn, ctask, sc->request_buffer, + sc->request_bufflen, + tcp_ctask->data_offset); if (rc == -EAGAIN) return rc; if (conn->datadgst_en) - iscsi_recv_digest_update(conn, sc->request_buffer, i); + iscsi_recv_digest_update(tcp_conn, sc->request_buffer, + i); rc = 0; goto done; } - offset = ctask->data_offset; + offset = tcp_ctask->data_offset; sg = sc->request_buffer; - if (ctask->data_offset) - for (i = 0; i < ctask->sg_count; i++) + if (tcp_ctask->data_offset) + for (i = 0; i < tcp_ctask->sg_count; i++) offset -= sg[i].length; /* we've passed through partial sg*/ if (offset < 0) offset = 0; - for (i = ctask->sg_count; i < sc->use_sg; i++) { + for (i = tcp_ctask->sg_count; i < sc->use_sg; i++) { char *dest; dest = kmap_atomic(sg[i].page, KM_SOFTIRQ0); - rc = iscsi_ctask_copy(conn, ctask, dest + sg[i].offset, + rc = iscsi_ctask_copy(tcp_conn, ctask, dest + sg[i].offset, sg[i].length, offset); kunmap_atomic(dest, KM_SOFTIRQ0); if (rc == -EAGAIN) @@ -952,15 +740,17 @@ static int iscsi_scsi_data_in(struct isc if (!rc) { if (conn->datadgst_en) { if (!offset) - crypto_digest_update(conn->data_rx_tfm, - &sg[i], 1); + crypto_digest_update( + tcp_conn->data_rx_tfm, + &sg[i], 1); else - partial_sg_digest_update(conn, &sg[i], + partial_sg_digest_update(tcp_conn, + &sg[i], sg[i].offset + offset, sg[i].length - offset); } offset = 0; - ctask->sg_count++; + tcp_ctask->sg_count++; } if (!ctask->data_count) { @@ -968,25 +758,26 @@ static int iscsi_scsi_data_in(struct isc /* * data-in is complete, but buffer not... */ - partial_sg_digest_update(conn, &sg[i], + partial_sg_digest_update(tcp_conn, &sg[i], sg[i].offset, sg[i].length-rc); rc = 0; break; } - if (!conn->in.copy) + if (!tcp_conn->in.copy) return -EAGAIN; } BUG_ON(ctask->data_count); done: /* check for non-exceptional status */ - if (conn->in.flags & ISCSI_FLAG_DATA_STATUS) { + if (tcp_conn->in.hdr->flags & ISCSI_FLAG_DATA_STATUS) { debug_scsi("done [sc %lx res %d itt 0x%x]\n", (long)sc, sc->result, ctask->itt); - conn->scsirsp_pdus_cnt++; - iscsi_ctask_cleanup(conn, ctask); - sc->scsi_done(sc); + spin_lock(&conn->session->lock); + __iscsi_ctask_cleanup(conn, ctask); + __iscsi_complete_pdu(conn, tcp_conn->in.hdr, NULL, 0); + spin_unlock(&conn->session->lock); } return rc; @@ -995,71 +786,38 @@ done: static int iscsi_data_recv(struct iscsi_conn *conn) { - struct iscsi_session *session = conn->session; - int rc = 0; + struct iscsi_tcp_conn *tcp_conn = conn->dd_data; + int rc = 0, opcode; - switch(conn->in.opcode) { + opcode = tcp_conn->in.hdr->opcode & ISCSI_OPCODE_MASK; + switch (opcode) { case ISCSI_OP_SCSI_DATA_IN: rc = iscsi_scsi_data_in(conn); break; - case ISCSI_OP_SCSI_CMD_RSP: { - /* - * SCSI Sense Data: - * copying the entire Data Segment. - */ - if (iscsi_tcp_copy(conn, conn->data, conn->in.datalen)) { - rc = -EAGAIN; - goto exit; - } - - /* - * check for sense - */ - conn->in.hdr = &conn->hdr; - conn->senselen = (conn->data[0] << 8) | conn->data[1]; - rc = iscsi_cmd_rsp(conn, conn->in.ctask); - if (!rc && conn->datadgst_en) - iscsi_recv_digest_update(conn, conn->data, - conn->in.datalen); - } - break; + case ISCSI_OP_SCSI_CMD_RSP: + spin_lock(&conn->session->lock); + __iscsi_ctask_cleanup(conn, tcp_conn->in.ctask); + spin_unlock(&conn->session->lock); case ISCSI_OP_TEXT_RSP: case ISCSI_OP_LOGIN_RSP: - case ISCSI_OP_NOOP_IN: { - struct iscsi_mgmt_task *mtask = NULL; - - if (conn->in.itt != ISCSI_RESERVED_TAG) - mtask = (struct iscsi_mgmt_task *) - session->mgmt_cmds[conn->in.itt - - ISCSI_MGMT_ITT_OFFSET]; - + case ISCSI_OP_NOOP_IN: + case ISCSI_OP_ASYNC_EVENT: + case ISCSI_OP_REJECT: /* * Collect data segment to the connection's data * placeholder */ - if (iscsi_tcp_copy(conn, conn->data, conn->in.datalen)) { + if (iscsi_tcp_copy(tcp_conn)) { rc = -EAGAIN; goto exit; } - rc = iscsi_recv_pdu(conn->cls_conn, conn->in.hdr, - conn->data, conn->in.datalen); - - if (!rc && conn->datadgst_en && - conn->in.opcode != ISCSI_OP_LOGIN_RSP) - iscsi_recv_digest_update(conn, conn->data, - conn->in.datalen); - - if (mtask && conn->login_mtask != mtask) { - spin_lock(&session->lock); - __kfifo_put(session->mgmtpool.queue, (void*)&mtask, - sizeof(void*)); - spin_unlock(&session->lock); - } - } - break; - case ISCSI_OP_ASYNC_EVENT: - case ISCSI_OP_REJECT: + rc = iscsi_complete_pdu(conn, tcp_conn->in.hdr, tcp_conn->data, + tcp_conn->in.datalen); + if (!rc && conn->datadgst_en && opcode != ISCSI_OP_LOGIN_RSP) + iscsi_recv_digest_update(tcp_conn, tcp_conn->data, + tcp_conn->in.datalen); + break; default: BUG_ON(1); } @@ -1080,6 +838,7 @@ iscsi_tcp_data_recv(read_descriptor_t *r { int rc; struct iscsi_conn *conn = rd_desc->arg.data; + struct iscsi_tcp_conn *tcp_conn = conn->dd_data; int processed; char pad[ISCSI_PAD_LEN]; struct scatterlist sg; @@ -1088,15 +847,15 @@ iscsi_tcp_data_recv(read_descriptor_t *r * Save current SKB and its offset in the corresponding * connection context. */ - conn->in.copy = skb->len - offset; - conn->in.offset = offset; - conn->in.skb = skb; - conn->in.len = conn->in.copy; - BUG_ON(conn->in.copy <= 0); - debug_tcp("in %d bytes\n", conn->in.copy); + tcp_conn->in.copy = skb->len - offset; + tcp_conn->in.offset = offset; + tcp_conn->in.skb = skb; + tcp_conn->in.len = tcp_conn->in.copy; + BUG_ON(tcp_conn->in.copy <= 0); + debug_tcp("in %d bytes\n", tcp_conn->in.copy); more: - conn->in.copied = 0; + tcp_conn->in.copied = 0; rc = 0; if (unlikely(conn->suspend_rx)) { @@ -1104,9 +863,9 @@ more: return 0; } - if (conn->in_progress == IN_PROGRESS_WAIT_HEADER || - conn->in_progress == IN_PROGRESS_HEADER_GATHER) { - rc = iscsi_hdr_extract(conn); + if (tcp_conn->in_progress == IN_PROGRESS_WAIT_HEADER || + tcp_conn->in_progress == IN_PROGRESS_HEADER_GATHER) { + rc = iscsi_hdr_extract(tcp_conn); if (rc) { if (rc == -EAGAIN) goto nomore; @@ -1119,90 +878,94 @@ more: /* * Verify and process incoming PDU header. */ - rc = iscsi_hdr_recv(conn); - if (!rc && conn->in.datalen) { + rc = iscsi_tcp_hdr_recv(conn); + if (!rc && tcp_conn->in.datalen) { if (conn->datadgst_en) { - BUG_ON(!conn->data_rx_tfm); - crypto_digest_init(conn->data_rx_tfm); + BUG_ON(!tcp_conn->data_rx_tfm); + crypto_digest_init(tcp_conn->data_rx_tfm); } - conn->in_progress = IN_PROGRESS_DATA_RECV; + tcp_conn->in_progress = IN_PROGRESS_DATA_RECV; } else if (rc) { iscsi_conn_failure(conn, rc); return 0; } } - if (conn->in_progress == IN_PROGRESS_DDIGEST_RECV) { + if (tcp_conn->in_progress == IN_PROGRESS_DDIGEST_RECV) { uint32_t recv_digest; + debug_tcp("extra data_recv offset %d copy %d\n", - conn->in.offset, conn->in.copy); - skb_copy_bits(conn->in.skb, conn->in.offset, + tcp_conn->in.offset, tcp_conn->in.copy); + skb_copy_bits(tcp_conn->in.skb, tcp_conn->in.offset, &recv_digest, 4); - conn->in.offset += 4; - conn->in.copy -= 4; - if (recv_digest != conn->in.datadgst) { + tcp_conn->in.offset += 4; + tcp_conn->in.copy -= 4; + if (recv_digest != tcp_conn->in.datadgst) { debug_tcp("iscsi_tcp: data digest error!" "0x%x != 0x%x\n", recv_digest, - conn->in.datadgst); + tcp_conn->in.datadgst); iscsi_conn_failure(conn, ISCSI_ERR_DATA_DGST); return 0; } else { debug_tcp("iscsi_tcp: data digest match!" "0x%x == 0x%x\n", recv_digest, - conn->in.datadgst); - conn->in_progress = IN_PROGRESS_WAIT_HEADER; + tcp_conn->in.datadgst); + tcp_conn->in_progress = IN_PROGRESS_WAIT_HEADER; } } - if (conn->in_progress == IN_PROGRESS_DATA_RECV && conn->in.copy) { + if (tcp_conn->in_progress == IN_PROGRESS_DATA_RECV && + tcp_conn->in.copy) { debug_tcp("data_recv offset %d copy %d\n", - conn->in.offset, conn->in.copy); + tcp_conn->in.offset, tcp_conn->in.copy); rc = iscsi_data_recv(conn); if (rc) { if (rc == -EAGAIN) { - rd_desc->count = conn->in.datalen - - conn->in.ctask->data_count; + rd_desc->count = tcp_conn->in.datalen - + tcp_conn->in.ctask->data_count; goto again; } iscsi_conn_failure(conn, rc); return 0; } - conn->in.copy -= conn->in.padding; - conn->in.offset += conn->in.padding; + tcp_conn->in.copy -= tcp_conn->in.padding; + tcp_conn->in.offset += tcp_conn->in.padding; if (conn->datadgst_en) { - if (conn->in.padding) { - debug_tcp("padding -> %d\n", conn->in.padding); - memset(pad, 0, conn->in.padding); - sg_init_one(&sg, pad, conn->in.padding); - crypto_digest_update(conn->data_rx_tfm, &sg, 1); - } - crypto_digest_final(conn->data_rx_tfm, - (u8 *) & conn->in.datadgst); - debug_tcp("rx digest 0x%x\n", conn->in.datadgst); - conn->in_progress = IN_PROGRESS_DDIGEST_RECV; + if (tcp_conn->in.padding) { + debug_tcp("padding -> %d\n", + tcp_conn->in.padding); + memset(pad, 0, tcp_conn->in.padding); + sg_init_one(&sg, pad, tcp_conn->in.padding); + crypto_digest_update(tcp_conn->data_rx_tfm, + &sg, 1); + } + crypto_digest_final(tcp_conn->data_rx_tfm, + (u8 *) & tcp_conn->in.datadgst); + debug_tcp("rx digest 0x%x\n", tcp_conn->in.datadgst); + tcp_conn->in_progress = IN_PROGRESS_DDIGEST_RECV; } else - conn->in_progress = IN_PROGRESS_WAIT_HEADER; + tcp_conn->in_progress = IN_PROGRESS_WAIT_HEADER; } debug_tcp("f, processed %d from out of %d padding %d\n", - conn->in.offset - offset, (int)len, conn->in.padding); - BUG_ON(conn->in.offset - offset > len); + tcp_conn->in.offset - offset, (int)len, tcp_conn->in.padding); + BUG_ON(tcp_conn->in.offset - offset > len); - if (conn->in.offset - offset != len) { + if (tcp_conn->in.offset - offset != len) { debug_tcp("continue to process %d bytes\n", - (int)len - (conn->in.offset - offset)); + (int)len - (tcp_conn->in.offset - offset)); goto more; } nomore: - processed = conn->in.offset - offset; + processed = tcp_conn->in.offset - offset; BUG_ON(processed == 0); return processed; again: - processed = conn->in.offset - offset; + processed = tcp_conn->in.offset - offset; debug_tcp("c, processed %d from out of %d rd_desc_cnt %d\n", processed, (int)len, (int)rd_desc->count); BUG_ON(processed == 0); @@ -1231,6 +994,7 @@ iscsi_tcp_data_ready(struct sock *sk, in static void iscsi_tcp_state_change(struct sock *sk) { + struct iscsi_tcp_conn *tcp_conn; struct iscsi_conn *conn; struct iscsi_session *session; void (*old_state_change)(struct sock *); @@ -1247,7 +1011,8 @@ iscsi_tcp_state_change(struct sock *sk) iscsi_conn_failure(conn, ISCSI_ERR_CONN_FAILED); } - old_state_change = conn->old_state_change; + tcp_conn = conn->dd_data; + old_state_change = tcp_conn->old_state_change; read_unlock(&sk->sk_callback_lock); @@ -1262,23 +1027,26 @@ static void iscsi_write_space(struct sock *sk) { struct iscsi_conn *conn = (struct iscsi_conn*)sk->sk_user_data; - conn->old_write_space(sk); + struct iscsi_tcp_conn *tcp_conn = conn->dd_data; + + tcp_conn->old_write_space(sk); debug_tcp("iscsi_write_space: cid %d\n", conn->id); - clear_bit(SUSPEND_BIT, &conn->suspend_tx); + clear_bit(ISCSI_SUSPEND_BIT, &conn->suspend_tx); scsi_queue_work(conn->session->host, &conn->xmitwork); } static void iscsi_conn_set_callbacks(struct iscsi_conn *conn) { - struct sock *sk = conn->sock->sk; + struct iscsi_tcp_conn *tcp_conn = conn->dd_data; + struct sock *sk = tcp_conn->sock->sk; /* assign new callbacks */ write_lock_bh(&sk->sk_callback_lock); sk->sk_user_data = conn; - conn->old_data_ready = sk->sk_data_ready; - conn->old_state_change = sk->sk_state_change; - conn->old_write_space = sk->sk_write_space; + tcp_conn->old_data_ready = sk->sk_data_ready; + tcp_conn->old_state_change = sk->sk_state_change; + tcp_conn->old_write_space = sk->sk_write_space; sk->sk_data_ready = iscsi_tcp_data_ready; sk->sk_state_change = iscsi_tcp_state_change; sk->sk_write_space = iscsi_write_space; @@ -1288,14 +1056,15 @@ iscsi_conn_set_callbacks(struct iscsi_co static void iscsi_conn_restore_callbacks(struct iscsi_conn *conn) { - struct sock *sk = conn->sock->sk; + struct iscsi_tcp_conn *tcp_conn = conn->dd_data; + struct sock *sk = tcp_conn->sock->sk; /* restore socket callbacks, see also: iscsi_conn_set_callbacks() */ write_lock_bh(&sk->sk_callback_lock); sk->sk_user_data = NULL; - sk->sk_data_ready = conn->old_data_ready; - sk->sk_state_change = conn->old_state_change; - sk->sk_write_space = conn->old_write_space; + sk->sk_data_ready = tcp_conn->old_data_ready; + sk->sk_state_change = tcp_conn->old_state_change; + sk->sk_write_space = tcp_conn->old_write_space; sk->sk_no_check = 0; write_unlock_bh(&sk->sk_callback_lock); } @@ -1310,7 +1079,8 @@ iscsi_conn_restore_callbacks(struct iscs static inline int iscsi_send(struct iscsi_conn *conn, struct iscsi_buf *buf, int size, int flags) { - struct socket *sk = conn->sock; + struct iscsi_tcp_conn *tcp_conn = conn->dd_data; + struct socket *sk = tcp_conn->sock; int offset = buf->sg.offset + buf->sent; /* @@ -1324,7 +1094,8 @@ iscsi_send(struct iscsi_conn *conn, stru if (buf->use_sendmsg) return sock_no_sendpage(sk, buf->sg.page, offset, size, flags); else - return conn->sendpage(sk, buf->sg.page, offset, size, flags); + return tcp_conn->sendpage(sk, buf->sg.page, offset, size, + flags); } /** @@ -1339,6 +1110,7 @@ iscsi_send(struct iscsi_conn *conn, stru static inline int iscsi_sendhdr(struct iscsi_conn *conn, struct iscsi_buf *buf, int datalen) { + struct iscsi_tcp_conn *tcp_conn; int flags = 0; /* MSG_DONTWAIT; */ int res, size; @@ -1356,8 +1128,9 @@ iscsi_sendhdr(struct iscsi_conn *conn, s return -EAGAIN; return 0; } else if (res == -EAGAIN) { - conn->sendpage_failures_cnt++; - set_bit(SUSPEND_BIT, &conn->suspend_tx); + tcp_conn = conn->dd_data; + tcp_conn->sendpage_failures_cnt++; + set_bit(ISCSI_SUSPEND_BIT, &conn->suspend_tx); } else if (res == -EPIPE) iscsi_conn_failure(conn, ISCSI_ERR_CONN_FAILED); @@ -1378,6 +1151,7 @@ static inline int iscsi_sendpage(struct iscsi_conn *conn, struct iscsi_buf *buf, int *count, int *sent) { + struct iscsi_tcp_conn *tcp_conn; int flags = 0; /* MSG_DONTWAIT; */ int res, size; @@ -1400,8 +1174,9 @@ iscsi_sendpage(struct iscsi_conn *conn, return -EAGAIN; return 0; } else if (res == -EAGAIN) { - conn->sendpage_failures_cnt++; - set_bit(SUSPEND_BIT, &conn->suspend_tx); + tcp_conn = conn->dd_data; + tcp_conn->sendpage_failures_cnt++; + set_bit(ISCSI_SUSPEND_BIT, &conn->suspend_tx); } else if (res == -EPIPE) iscsi_conn_failure(conn, ISCSI_ERR_CONN_FAILED); @@ -1409,30 +1184,35 @@ iscsi_sendpage(struct iscsi_conn *conn, } static inline void -iscsi_data_digest_init(struct iscsi_conn *conn, struct iscsi_cmd_task *ctask) +iscsi_data_digest_init(struct iscsi_tcp_conn *tcp_conn, + struct iscsi_cmd_task *ctask) { - BUG_ON(!conn->data_tx_tfm); - crypto_digest_init(conn->data_tx_tfm); - ctask->digest_count = 4; + struct iscsi_tcp_cmd_task *tcp_ctask = ctask->dd_data; + + BUG_ON(!tcp_conn->data_tx_tfm); + crypto_digest_init(tcp_conn->data_tx_tfm); + tcp_ctask->digest_count = 4; } static int iscsi_digest_final_send(struct iscsi_conn *conn, struct iscsi_cmd_task *ctask, struct iscsi_buf *buf, uint32_t *digest, int final) { + struct iscsi_tcp_cmd_task *tcp_ctask = ctask->dd_data; + struct iscsi_tcp_conn *tcp_conn = conn->dd_data; int rc = 0; int sent = 0; if (final) - crypto_digest_final(conn->data_tx_tfm, (u8*)digest); + crypto_digest_final(tcp_conn->data_tx_tfm, (u8*)digest); iscsi_buf_init_virt(buf, (char*)digest, 4); - rc = iscsi_sendpage(conn, buf, &ctask->digest_count, &sent); + rc = iscsi_sendpage(conn, buf, &tcp_ctask->digest_count, &sent); if (rc) { - ctask->datadigest = *digest; - ctask->xmstate |= XMSTATE_DATA_DIGEST; + tcp_ctask->datadigest = *digest; + tcp_ctask->xmstate |= XMSTATE_DATA_DIGEST; } else - ctask->digest_count = 4; + tcp_ctask->digest_count = 4; return rc; } @@ -1453,21 +1233,23 @@ static void iscsi_solicit_data_cont(struct iscsi_conn *conn, struct iscsi_cmd_task *ctask, struct iscsi_r2t_info *r2t, int left) { + struct iscsi_tcp_cmd_task *tcp_ctask = ctask->dd_data; struct iscsi_data *hdr; struct iscsi_data_task *dtask; struct scsi_cmnd *sc = ctask->sc; int new_offset; - dtask = mempool_alloc(ctask->datapool, GFP_ATOMIC); + dtask = mempool_alloc(tcp_ctask->datapool, GFP_ATOMIC); BUG_ON(!dtask); + INIT_LIST_HEAD(&dtask->item); hdr = &dtask->hdr; memset(hdr, 0, sizeof(struct iscsi_data)); hdr->ttt = r2t->ttt; hdr->datasn = cpu_to_be32(r2t->solicit_datasn); r2t->solicit_datasn++; hdr->opcode = ISCSI_OP_SCSI_DATA_OUT; - memcpy(hdr->lun, ctask->hdr.lun, sizeof(hdr->lun)); - hdr->itt = ctask->hdr.itt; + memcpy(hdr->lun, ctask->hdr->lun, sizeof(hdr->lun)); + hdr->itt = ctask->hdr->itt; hdr->exp_statsn = r2t->exp_statsn; new_offset = r2t->data_offset + r2t->sent; hdr->offset = cpu_to_be32(new_offset); @@ -1487,175 +1269,102 @@ iscsi_solicit_data_cont(struct iscsi_con r2t->dtask = dtask; if (sc->use_sg && !iscsi_buf_left(&r2t->sendbuf)) { - BUG_ON(ctask->bad_sg == r2t->sg); + BUG_ON(tcp_ctask->bad_sg == r2t->sg); iscsi_buf_init_sg(&r2t->sendbuf, r2t->sg); r2t->sg += 1; } else - iscsi_buf_init_iov(&ctask->sendbuf, + iscsi_buf_init_iov(&tcp_ctask->sendbuf, (char*)sc->request_buffer + new_offset, r2t->data_count); - list_add(&dtask->item, &ctask->dataqueue); + list_add(&dtask->item, &tcp_ctask->dataqueue); } static void iscsi_unsolicit_data_init(struct iscsi_conn *conn, struct iscsi_cmd_task *ctask) { - struct iscsi_data *hdr; + struct iscsi_tcp_cmd_task *tcp_ctask = ctask->dd_data; struct iscsi_data_task *dtask; - dtask = mempool_alloc(ctask->datapool, GFP_ATOMIC); + dtask = mempool_alloc(tcp_ctask->datapool, GFP_ATOMIC); BUG_ON(!dtask); - hdr = &dtask->hdr; - memset(hdr, 0, sizeof(struct iscsi_data)); - hdr->ttt = cpu_to_be32(ISCSI_RESERVED_TAG); - hdr->datasn = cpu_to_be32(ctask->unsol_datasn); - ctask->unsol_datasn++; - hdr->opcode = ISCSI_OP_SCSI_DATA_OUT; - memcpy(hdr->lun, ctask->hdr.lun, sizeof(hdr->lun)); - hdr->itt = ctask->hdr.itt; - hdr->exp_statsn = cpu_to_be32(conn->exp_statsn); - hdr->offset = cpu_to_be32(ctask->total_length - - ctask->r2t_data_count - - ctask->unsol_count); - if (ctask->unsol_count > conn->max_xmit_dlength) { - hton24(hdr->dlength, conn->max_xmit_dlength); - ctask->data_count = conn->max_xmit_dlength; - hdr->flags = 0; - } else { - hton24(hdr->dlength, ctask->unsol_count); - ctask->data_count = ctask->unsol_count; - hdr->flags = ISCSI_FLAG_CMD_FINAL; - } + INIT_LIST_HEAD(&dtask->item); - iscsi_buf_init_virt(&ctask->headbuf, (char*)hdr, + iscsi_prep_unsolicit_data_pdu(ctask, &dtask->hdr, + tcp_ctask->r2t_data_count); + iscsi_buf_init_virt(&tcp_ctask->headbuf, (char*)&dtask->hdr, sizeof(struct iscsi_hdr)); - list_add(&dtask->item, &ctask->dataqueue); - - ctask->dtask = dtask; + list_add(&dtask->item, &tcp_ctask->dataqueue); + tcp_ctask->dtask = dtask; } /** - * iscsi_cmd_init - Initialize iSCSI SCSI_READ or SCSI_WRITE commands + * iscsi_tcp_cmd_init - Initialize iSCSI SCSI_READ or SCSI_WRITE commands * @conn: iscsi connection * @ctask: scsi command task * @sc: scsi command **/ static void -iscsi_cmd_init(struct iscsi_conn *conn, struct iscsi_cmd_task *ctask, - struct scsi_cmnd *sc) +iscsi_tcp_cmd_init(struct iscsi_cmd_task *ctask) { - struct iscsi_session *session = conn->session; - - BUG_ON(__kfifo_len(ctask->r2tqueue)); + struct scsi_cmnd *sc = ctask->sc; + struct iscsi_tcp_cmd_task *tcp_ctask = ctask->dd_data; - ctask->sc = sc; - ctask->conn = conn; - ctask->hdr.opcode = ISCSI_OP_SCSI_CMD; - ctask->hdr.flags = ISCSI_ATTR_SIMPLE; - int_to_scsilun(sc->device->lun, (struct scsi_lun *)ctask->hdr.lun); - ctask->hdr.itt = ctask->itt | (conn->id << CID_SHIFT) | - (session->age << AGE_SHIFT); - ctask->hdr.data_length = cpu_to_be32(sc->request_bufflen); - ctask->hdr.cmdsn = cpu_to_be32(session->cmdsn); session->cmdsn++; - ctask->hdr.exp_statsn = cpu_to_be32(conn->exp_statsn); - memcpy(ctask->hdr.cdb, sc->cmnd, sc->cmd_len); - memset(&ctask->hdr.cdb[sc->cmd_len], 0, MAX_COMMAND_SIZE - sc->cmd_len); - - ctask->mtask = NULL; - ctask->sent = 0; - ctask->sg_count = 0; + BUG_ON(__kfifo_len(tcp_ctask->r2tqueue)); - ctask->total_length = sc->request_bufflen; + tcp_ctask->sent = 0; + tcp_ctask->sg_count = 0; if (sc->sc_data_direction == DMA_TO_DEVICE) { - ctask->exp_r2tsn = 0; - ctask->hdr.flags |= ISCSI_FLAG_CMD_WRITE; + tcp_ctask->xmstate = XMSTATE_W_HDR; + tcp_ctask->exp_r2tsn = 0; BUG_ON(ctask->total_length == 0); + if (sc->use_sg) { struct scatterlist *sg = sc->request_buffer; - iscsi_buf_init_sg(&ctask->sendbuf, - &sg[ctask->sg_count++]); - ctask->sg = sg; - ctask->bad_sg = sg + sc->use_sg; - } else { - iscsi_buf_init_iov(&ctask->sendbuf, sc->request_buffer, - sc->request_bufflen); - } - - /* - * Write counters: - * - * imm_count bytes to be sent right after - * SCSI PDU Header - * - * unsol_count bytes(as Data-Out) to be sent - * without R2T ack right after - * immediate data - * - * r2t_data_count bytes to be sent via R2T ack's - * - * pad_count bytes to be sent as zero-padding - */ - ctask->imm_count = 0; - ctask->unsol_count = 0; - ctask->unsol_datasn = 0; - ctask->xmstate = XMSTATE_W_HDR; - /* calculate write padding */ - ctask->pad_count = ctask->total_length & (ISCSI_PAD_LEN-1); - if (ctask->pad_count) { - ctask->pad_count = ISCSI_PAD_LEN - ctask->pad_count; + iscsi_buf_init_sg(&tcp_ctask->sendbuf, + &sg[tcp_ctask->sg_count++]); + tcp_ctask->sg = sg; + tcp_ctask->bad_sg = sg + sc->use_sg; + } else + iscsi_buf_init_iov(&tcp_ctask->sendbuf, + sc->request_buffer, + sc->request_bufflen); + + if (ctask->imm_count) + tcp_ctask->xmstate |= XMSTATE_IMM_DATA; + + tcp_ctask->pad_count = ctask->total_length & (ISCSI_PAD_LEN-1); + if (tcp_ctask->pad_count) { + tcp_ctask->pad_count = ISCSI_PAD_LEN - + tcp_ctask->pad_count; debug_scsi("write padding %d bytes\n", - ctask->pad_count); - ctask->xmstate |= XMSTATE_W_PAD; + tcp_ctask->pad_count); + tcp_ctask->xmstate |= XMSTATE_W_PAD; } - if (session->imm_data_en) { - if (ctask->total_length >= session->first_burst) - ctask->imm_count = min(session->first_burst, - conn->max_xmit_dlength); - else - ctask->imm_count = min(ctask->total_length, - conn->max_xmit_dlength); - hton24(ctask->hdr.dlength, ctask->imm_count); - ctask->xmstate |= XMSTATE_IMM_DATA; - } else - zero_data(ctask->hdr.dlength); - - if (!session->initial_r2t_en) - ctask->unsol_count = min(session->first_burst, - ctask->total_length) - ctask->imm_count; - if (!ctask->unsol_count) - /* No unsolicit Data-Out's */ - ctask->hdr.flags |= ISCSI_FLAG_CMD_FINAL; - else - ctask->xmstate |= XMSTATE_UNS_HDR | XMSTATE_UNS_INIT; - ctask->r2t_data_count = ctask->total_length - + if (ctask->unsol_count) + tcp_ctask->xmstate |= XMSTATE_UNS_HDR | + XMSTATE_UNS_INIT; + tcp_ctask->r2t_data_count = ctask->total_length - ctask->imm_count - ctask->unsol_count; debug_scsi("cmd [itt %x total %d imm %d imm_data %d " "r2t_data %d]\n", ctask->itt, ctask->total_length, ctask->imm_count, - ctask->unsol_count, ctask->r2t_data_count); - } else { - ctask->hdr.flags |= ISCSI_FLAG_CMD_FINAL; - if (sc->sc_data_direction == DMA_FROM_DEVICE) - ctask->hdr.flags |= ISCSI_FLAG_CMD_READ; - ctask->datasn = 0; - ctask->xmstate = XMSTATE_R_HDR; - zero_data(ctask->hdr.dlength); - } + ctask->unsol_count, tcp_ctask->r2t_data_count); + } else + tcp_ctask->xmstate = XMSTATE_R_HDR; - iscsi_buf_init_virt(&ctask->headbuf, (char*)&ctask->hdr, + iscsi_buf_init_virt(&tcp_ctask->headbuf, (char*)ctask->hdr, sizeof(struct iscsi_hdr)); - conn->scsicmd_pdus_cnt++; } /** - * iscsi_mtask_xmit - xmit management(immediate) task + * iscsi_tcp_mtask_xmit - xmit management(immediate) task * @conn: iscsi connection * @mtask: task management task * @@ -1669,70 +1378,87 @@ iscsi_cmd_init(struct iscsi_conn *conn, * IN_PROGRESS_IMM_DATA - PDU Data xmit in progress **/ static int -iscsi_mtask_xmit(struct iscsi_conn *conn, struct iscsi_mgmt_task *mtask) +iscsi_tcp_mtask_xmit(struct iscsi_conn *conn, struct iscsi_mgmt_task *mtask) { + struct iscsi_tcp_mgmt_task *tcp_mtask = mtask->dd_data; debug_scsi("mtask deq [cid %d state %x itt 0x%x]\n", - conn->id, mtask->xmstate, mtask->itt); + conn->id, tcp_mtask->xmstate, mtask->itt); - if (mtask->xmstate & XMSTATE_IMM_HDR) { - mtask->xmstate &= ~XMSTATE_IMM_HDR; + if (tcp_mtask->xmstate & XMSTATE_IMM_HDR) { + tcp_mtask->xmstate &= ~XMSTATE_IMM_HDR; if (mtask->data_count) - mtask->xmstate |= XMSTATE_IMM_DATA; + tcp_mtask->xmstate |= XMSTATE_IMM_DATA; if (conn->c_stage != ISCSI_CONN_INITIAL_STAGE && - conn->stop_stage != STOP_CONN_RECOVER && + conn->stop_stage != STOP_CONN_RECOVER && conn->hdrdgst_en) - iscsi_hdr_digest(conn, &mtask->headbuf, - (u8*)mtask->hdrext); - if (iscsi_sendhdr(conn, &mtask->headbuf, mtask->data_count)) { - mtask->xmstate |= XMSTATE_IMM_HDR; + iscsi_hdr_digest(conn, &tcp_mtask->headbuf, + (u8*)tcp_mtask->hdrext); + if (iscsi_sendhdr(conn, &tcp_mtask->headbuf, + mtask->data_count)) { + tcp_mtask->xmstate |= XMSTATE_IMM_HDR; if (mtask->data_count) - mtask->xmstate &= ~XMSTATE_IMM_DATA; + tcp_mtask->xmstate &= ~XMSTATE_IMM_DATA; return -EAGAIN; } } - if (mtask->xmstate & XMSTATE_IMM_DATA) { + if (tcp_mtask->xmstate & XMSTATE_IMM_DATA) { BUG_ON(!mtask->data_count); - mtask->xmstate &= ~XMSTATE_IMM_DATA; + tcp_mtask->xmstate &= ~XMSTATE_IMM_DATA; /* FIXME: implement. * Virtual buffer could be spreaded across multiple pages... */ do { - if (iscsi_sendpage(conn, &mtask->sendbuf, - &mtask->data_count, &mtask->sent)) { - mtask->xmstate |= XMSTATE_IMM_DATA; + if (iscsi_sendpage(conn, &tcp_mtask->sendbuf, + &mtask->data_count, &tcp_mtask->sent)) { + tcp_mtask->xmstate |= XMSTATE_IMM_DATA; return -EAGAIN; } } while (mtask->data_count); } - BUG_ON(mtask->xmstate != XMSTATE_IDLE); + BUG_ON(tcp_mtask->xmstate != XMSTATE_IDLE); + if (mtask->hdr->itt == cpu_to_be32(ISCSI_RESERVED_TAG)) { + struct iscsi_session *session = conn->session; + + spin_lock_bh(&session->lock); + list_del(&conn->mtask->running); + __kfifo_put(session->mgmtpool.queue, (void*)&conn->mtask, + sizeof(void*)); + spin_unlock_bh(&session->lock); + } return 0; } static inline int -handle_xmstate_r_hdr(struct iscsi_conn *conn, struct iscsi_cmd_task *ctask) +handle_xmstate_r_hdr(struct iscsi_conn *conn, + struct iscsi_tcp_cmd_task *tcp_ctask) { - ctask->xmstate &= ~XMSTATE_R_HDR; + tcp_ctask->xmstate &= ~XMSTATE_R_HDR; if (conn->hdrdgst_en) - iscsi_hdr_digest(conn, &ctask->headbuf, (u8*)ctask->hdrext); - if (!iscsi_sendhdr(conn, &ctask->headbuf, 0)) { - BUG_ON(ctask->xmstate != XMSTATE_IDLE); + iscsi_hdr_digest(conn, &tcp_ctask->headbuf, + (u8*)tcp_ctask->hdrext); + if (!iscsi_sendhdr(conn, &tcp_ctask->headbuf, 0)) { + BUG_ON(tcp_ctask->xmstate != XMSTATE_IDLE); return 0; /* wait for Data-In */ } - ctask->xmstate |= XMSTATE_R_HDR; + tcp_ctask->xmstate |= XMSTATE_R_HDR; return -EAGAIN; } static inline int -handle_xmstate_w_hdr(struct iscsi_conn *conn, struct iscsi_cmd_task *ctask) +handle_xmstate_w_hdr(struct iscsi_conn *conn, + struct iscsi_cmd_task *ctask) { - ctask->xmstate &= ~XMSTATE_W_HDR; + struct iscsi_tcp_cmd_task *tcp_ctask = ctask->dd_data; + + tcp_ctask->xmstate &= ~XMSTATE_W_HDR; if (conn->hdrdgst_en) - iscsi_hdr_digest(conn, &ctask->headbuf, (u8*)ctask->hdrext); - if (iscsi_sendhdr(conn, &ctask->headbuf, ctask->imm_count)) { - ctask->xmstate |= XMSTATE_W_HDR; + iscsi_hdr_digest(conn, &tcp_ctask->headbuf, + (u8*)tcp_ctask->hdrext); + if (iscsi_sendhdr(conn, &tcp_ctask->headbuf, ctask->imm_count)) { + tcp_ctask->xmstate |= XMSTATE_W_HDR; return -EAGAIN; } return 0; @@ -1742,13 +1468,15 @@ static inline int handle_xmstate_data_digest(struct iscsi_conn *conn, struct iscsi_cmd_task *ctask) { - ctask->xmstate &= ~XMSTATE_DATA_DIGEST; - debug_tcp("resent data digest 0x%x\n", ctask->datadigest); - if (iscsi_digest_final_send(conn, ctask, &ctask->immbuf, - &ctask->datadigest, 0)) { - ctask->xmstate |= XMSTATE_DATA_DIGEST; + struct iscsi_tcp_cmd_task *tcp_ctask = ctask->dd_data; + + tcp_ctask->xmstate &= ~XMSTATE_DATA_DIGEST; + debug_tcp("resent data digest 0x%x\n", tcp_ctask->datadigest); + if (iscsi_digest_final_send(conn, ctask, &tcp_ctask->immbuf, + &tcp_ctask->datadigest, 0)) { + tcp_ctask->xmstate |= XMSTATE_DATA_DIGEST; debug_tcp("resent data digest 0x%x fail!\n", - ctask->datadigest); + tcp_ctask->datadigest); return -EAGAIN; } return 0; @@ -1757,44 +1485,47 @@ handle_xmstate_data_digest(struct iscsi_ static inline int handle_xmstate_imm_data(struct iscsi_conn *conn, struct iscsi_cmd_task *ctask) { + struct iscsi_tcp_cmd_task *tcp_ctask = ctask->dd_data; + struct iscsi_tcp_conn *tcp_conn = conn->dd_data; + BUG_ON(!ctask->imm_count); - ctask->xmstate &= ~XMSTATE_IMM_DATA; + tcp_ctask->xmstate &= ~XMSTATE_IMM_DATA; if (conn->datadgst_en) { - iscsi_data_digest_init(conn, ctask); - ctask->immdigest = 0; + iscsi_data_digest_init(tcp_conn, ctask); + tcp_ctask->immdigest = 0; } for (;;) { - if (iscsi_sendpage(conn, &ctask->sendbuf, &ctask->imm_count, - &ctask->sent)) { - ctask->xmstate |= XMSTATE_IMM_DATA; + if (iscsi_sendpage(conn, &tcp_ctask->sendbuf, &ctask->imm_count, + &tcp_ctask->sent)) { + tcp_ctask->xmstate |= XMSTATE_IMM_DATA; if (conn->datadgst_en) { - crypto_digest_final(conn->data_tx_tfm, - (u8*)&ctask->immdigest); + crypto_digest_final(tcp_conn->data_tx_tfm, + (u8*)&tcp_ctask->immdigest); debug_tcp("tx imm sendpage fail 0x%x\n", - ctask->datadigest); + tcp_ctask->datadigest); } return -EAGAIN; } if (conn->datadgst_en) - crypto_digest_update(conn->data_tx_tfm, - &ctask->sendbuf.sg, 1); + crypto_digest_update(tcp_conn->data_tx_tfm, + &tcp_ctask->sendbuf.sg, 1); if (!ctask->imm_count) break; - iscsi_buf_init_sg(&ctask->sendbuf, - &ctask->sg[ctask->sg_count++]); + iscsi_buf_init_sg(&tcp_ctask->sendbuf, + &tcp_ctask->sg[tcp_ctask->sg_count++]); } - if (conn->datadgst_en && !(ctask->xmstate & XMSTATE_W_PAD)) { - if (iscsi_digest_final_send(conn, ctask, &ctask->immbuf, - &ctask->immdigest, 1)) { + if (conn->datadgst_en && !(tcp_ctask->xmstate & XMSTATE_W_PAD)) { + if (iscsi_digest_final_send(conn, ctask, &tcp_ctask->immbuf, + &tcp_ctask->immdigest, 1)) { debug_tcp("sending imm digest 0x%x fail!\n", - ctask->immdigest); + tcp_ctask->immdigest); return -EAGAIN; } - debug_tcp("sending imm digest 0x%x\n", ctask->immdigest); + debug_tcp("sending imm digest 0x%x\n", tcp_ctask->immdigest); } return 0; @@ -1803,52 +1534,55 @@ handle_xmstate_imm_data(struct iscsi_con static inline int handle_xmstate_uns_hdr(struct iscsi_conn *conn, struct iscsi_cmd_task *ctask) { + struct iscsi_tcp_cmd_task *tcp_ctask = ctask->dd_data; struct iscsi_data_task *dtask; - ctask->xmstate |= XMSTATE_UNS_DATA; - if (ctask->xmstate & XMSTATE_UNS_INIT) { + tcp_ctask->xmstate |= XMSTATE_UNS_DATA; + if (tcp_ctask->xmstate & XMSTATE_UNS_INIT) { iscsi_unsolicit_data_init(conn, ctask); - BUG_ON(!ctask->dtask); - dtask = ctask->dtask; + BUG_ON(!tcp_ctask->dtask); + dtask = tcp_ctask->dtask; if (conn->hdrdgst_en) - iscsi_hdr_digest(conn, &ctask->headbuf, + iscsi_hdr_digest(conn, &tcp_ctask->headbuf, (u8*)dtask->hdrext); - ctask->xmstate &= ~XMSTATE_UNS_INIT; + tcp_ctask->xmstate &= ~XMSTATE_UNS_INIT; } - if (iscsi_sendhdr(conn, &ctask->headbuf, ctask->data_count)) { - ctask->xmstate &= ~XMSTATE_UNS_DATA; - ctask->xmstate |= XMSTATE_UNS_HDR; + if (iscsi_sendhdr(conn, &tcp_ctask->headbuf, ctask->data_count)) { + tcp_ctask->xmstate &= ~XMSTATE_UNS_DATA; + tcp_ctask->xmstate |= XMSTATE_UNS_HDR; return -EAGAIN; } debug_scsi("uns dout [itt 0x%x dlen %d sent %d]\n", - ctask->itt, ctask->unsol_count, ctask->sent); + ctask->itt, ctask->unsol_count, tcp_ctask->sent); return 0; } static inline int handle_xmstate_uns_data(struct iscsi_conn *conn, struct iscsi_cmd_task *ctask) { - struct iscsi_data_task *dtask = ctask->dtask; + struct iscsi_tcp_cmd_task *tcp_ctask = ctask->dd_data; + struct iscsi_data_task *dtask = tcp_ctask->dtask; + struct iscsi_tcp_conn *tcp_conn = conn->dd_data; BUG_ON(!ctask->data_count); - ctask->xmstate &= ~XMSTATE_UNS_DATA; + tcp_ctask->xmstate &= ~XMSTATE_UNS_DATA; if (conn->datadgst_en) { - iscsi_data_digest_init(conn, ctask); + iscsi_data_digest_init(tcp_conn, ctask); dtask->digest = 0; } for (;;) { - int start = ctask->sent; + int start = tcp_ctask->sent; - if (iscsi_sendpage(conn, &ctask->sendbuf, &ctask->data_count, - &ctask->sent)) { - ctask->unsol_count -= ctask->sent - start; - ctask->xmstate |= XMSTATE_UNS_DATA; + if (iscsi_sendpage(conn, &tcp_ctask->sendbuf, + &ctask->data_count, &tcp_ctask->sent)) { + ctask->unsol_count -= tcp_ctask->sent - start; + tcp_ctask->xmstate |= XMSTATE_UNS_DATA; /* will continue with this ctask later.. */ if (conn->datadgst_en) { - crypto_digest_final(conn->data_tx_tfm, + crypto_digest_final(tcp_conn->data_tx_tfm, (u8 *)&dtask->digest); debug_tcp("tx uns data fail 0x%x\n", dtask->digest); @@ -1856,21 +1590,21 @@ handle_xmstate_uns_data(struct iscsi_con return -EAGAIN; } - BUG_ON(ctask->sent > ctask->total_length); - ctask->unsol_count -= ctask->sent - start; + BUG_ON(tcp_ctask->sent > ctask->total_length); + ctask->unsol_count -= tcp_ctask->sent - start; /* * XXX:we may run here with un-initial sendbuf. * so pass it */ - if (conn->datadgst_en && ctask->sent - start > 0) - crypto_digest_update(conn->data_tx_tfm, - &ctask->sendbuf.sg, 1); + if (conn->datadgst_en && tcp_ctask->sent - start > 0) + crypto_digest_update(tcp_conn->data_tx_tfm, + &tcp_ctask->sendbuf.sg, 1); if (!ctask->data_count) break; - iscsi_buf_init_sg(&ctask->sendbuf, - &ctask->sg[ctask->sg_count++]); + iscsi_buf_init_sg(&tcp_ctask->sendbuf, + &tcp_ctask->sg[tcp_ctask->sg_count++]); } BUG_ON(ctask->unsol_count < 0); @@ -1890,11 +1624,11 @@ handle_xmstate_uns_data(struct iscsi_con debug_tcp("sending uns digest 0x%x, more uns\n", dtask->digest); } - ctask->xmstate |= XMSTATE_UNS_INIT; + tcp_ctask->xmstate |= XMSTATE_UNS_INIT; return 1; } - if (conn->datadgst_en && !(ctask->xmstate & XMSTATE_W_PAD)) { + if (conn->datadgst_en && !(tcp_ctask->xmstate & XMSTATE_W_PAD)) { if (iscsi_digest_final_send(conn, ctask, &dtask->digestbuf, &dtask->digest, 1)) { @@ -1912,15 +1646,17 @@ static inline int handle_xmstate_sol_data(struct iscsi_conn *conn, struct iscsi_cmd_task *ctask) { struct iscsi_session *session = conn->session; - struct iscsi_r2t_info *r2t = ctask->r2t; + struct iscsi_tcp_conn *tcp_conn = conn->dd_data; + struct iscsi_tcp_cmd_task *tcp_ctask = ctask->dd_data; + struct iscsi_r2t_info *r2t = tcp_ctask->r2t; struct iscsi_data_task *dtask = r2t->dtask; int left; - ctask->xmstate &= ~XMSTATE_SOL_DATA; - ctask->dtask = dtask; + tcp_ctask->xmstate &= ~XMSTATE_SOL_DATA; + tcp_ctask->dtask = dtask; if (conn->datadgst_en) { - iscsi_data_digest_init(conn, ctask); + iscsi_data_digest_init(tcp_conn, ctask); dtask->digest = 0; } solicit_again: @@ -1931,10 +1667,10 @@ solicit_again: goto data_out_done; if (iscsi_sendpage(conn, &r2t->sendbuf, &r2t->data_count, &r2t->sent)) { - ctask->xmstate |= XMSTATE_SOL_DATA; + tcp_ctask->xmstate |= XMSTATE_SOL_DATA; /* will continue with this ctask later.. */ if (conn->datadgst_en) { - crypto_digest_final(conn->data_tx_tfm, + crypto_digest_final(tcp_conn->data_tx_tfm, (u8 *)&dtask->digest); debug_tcp("r2t data send fail 0x%x\n", dtask->digest); } @@ -1943,12 +1679,13 @@ solicit_again: BUG_ON(r2t->data_count < 0); if (conn->datadgst_en) - crypto_digest_update(conn->data_tx_tfm, &r2t->sendbuf.sg, 1); + crypto_digest_update(tcp_conn->data_tx_tfm, &r2t->sendbuf.sg, + 1); if (r2t->data_count) { BUG_ON(ctask->sc->use_sg == 0); if (!iscsi_buf_left(&r2t->sendbuf)) { - BUG_ON(ctask->bad_sg == r2t->sg); + BUG_ON(tcp_ctask->bad_sg == r2t->sg); iscsi_buf_init_sg(&r2t->sendbuf, r2t->sg); r2t->sg += 1; } @@ -1975,8 +1712,8 @@ data_out_done: dtask->digest); } iscsi_solicit_data_cont(conn, ctask, r2t, left); - ctask->xmstate |= XMSTATE_SOL_DATA; - ctask->xmstate &= ~XMSTATE_SOL_HDR; + tcp_ctask->xmstate |= XMSTATE_SOL_DATA; + tcp_ctask->xmstate &= ~XMSTATE_SOL_HDR; return 1; } @@ -1984,7 +1721,7 @@ data_out_done: * Done with this R2T. Check if there are more * outstanding R2Ts ready to be processed. */ - BUG_ON(ctask->r2t_data_count - r2t->data_length < 0); + BUG_ON(tcp_ctask->r2t_data_count - r2t->data_length < 0); if (conn->datadgst_en) { if (iscsi_digest_final_send(conn, ctask, &dtask->digestbuf, &dtask->digest, 1)) { @@ -1995,15 +1732,15 @@ data_out_done: debug_tcp("r2t done dout digest 0x%x\n", dtask->digest); } - ctask->r2t_data_count -= r2t->data_length; - ctask->r2t = NULL; + tcp_ctask->r2t_data_count -= r2t->data_length; + tcp_ctask->r2t = NULL; spin_lock_bh(&session->lock); - __kfifo_put(ctask->r2tpool.queue, (void*)&r2t, sizeof(void*)); + __kfifo_put(tcp_ctask->r2tpool.queue, (void*)&r2t, sizeof(void*)); spin_unlock_bh(&session->lock); - if (__kfifo_get(ctask->r2tqueue, (void*)&r2t, sizeof(void*))) { - ctask->r2t = r2t; - ctask->xmstate |= XMSTATE_SOL_DATA; - ctask->xmstate &= ~XMSTATE_SOL_HDR; + if (__kfifo_get(tcp_ctask->r2tqueue, (void*)&r2t, sizeof(void*))) { + tcp_ctask->r2t = r2t; + tcp_ctask->xmstate |= XMSTATE_SOL_DATA; + tcp_ctask->xmstate &= ~XMSTATE_SOL_HDR; return 1; } @@ -2013,29 +1750,34 @@ data_out_done: static inline int handle_xmstate_w_pad(struct iscsi_conn *conn, struct iscsi_cmd_task *ctask) { - struct iscsi_data_task *dtask = ctask->dtask; + struct iscsi_tcp_cmd_task *tcp_ctask = ctask->dd_data; + struct iscsi_tcp_conn *tcp_conn = conn->dd_data; + struct iscsi_data_task *dtask = tcp_ctask->dtask; int sent; - ctask->xmstate &= ~XMSTATE_W_PAD; - iscsi_buf_init_virt(&ctask->sendbuf, (char*)&ctask->pad, - ctask->pad_count); - if (iscsi_sendpage(conn, &ctask->sendbuf, &ctask->pad_count, &sent)) { - ctask->xmstate |= XMSTATE_W_PAD; + tcp_ctask->xmstate &= ~XMSTATE_W_PAD; + iscsi_buf_init_virt(&tcp_ctask->sendbuf, (char*)&tcp_ctask->pad, + tcp_ctask->pad_count); + if (iscsi_sendpage(conn, &tcp_ctask->sendbuf, &tcp_ctask->pad_count, + &sent)) { + tcp_ctask->xmstate |= XMSTATE_W_PAD; return -EAGAIN; } if (conn->datadgst_en) { - crypto_digest_update(conn->data_tx_tfm, &ctask->sendbuf.sg, 1); + crypto_digest_update(tcp_conn->data_tx_tfm, + &tcp_ctask->sendbuf.sg, 1); /* imm data? */ if (!dtask) { - if (iscsi_digest_final_send(conn, ctask, &ctask->immbuf, - &ctask->immdigest, 1)) { + if (iscsi_digest_final_send(conn, ctask, + &tcp_ctask->immbuf, + &tcp_ctask->immdigest, 1)) { debug_tcp("send padding digest 0x%x" - "fail!\n", ctask->immdigest); + "fail!\n", tcp_ctask->immdigest); return -EAGAIN; } debug_tcp("done with padding, digest 0x%x\n", - ctask->datadigest); + tcp_ctask->datadigest); } else { if (iscsi_digest_final_send(conn, ctask, &dtask->digestbuf, @@ -2053,12 +1795,13 @@ handle_xmstate_w_pad(struct iscsi_conn * } static int -iscsi_ctask_xmit(struct iscsi_conn *conn, struct iscsi_cmd_task *ctask) +iscsi_tcp_ctask_xmit(struct iscsi_conn *conn, struct iscsi_cmd_task *ctask) { + struct iscsi_tcp_cmd_task *tcp_ctask = ctask->dd_data; int rc = 0; debug_scsi("ctask deq [cid %d xmstate %x itt 0x%x]\n", - conn->id, ctask->xmstate, ctask->itt); + conn->id, tcp_ctask->xmstate, ctask->itt); /* * serialize with TMF AbortTask @@ -2066,40 +1809,40 @@ iscsi_ctask_xmit(struct iscsi_conn *conn if (ctask->mtask) return rc; - if (ctask->xmstate & XMSTATE_R_HDR) { - rc = handle_xmstate_r_hdr(conn, ctask); + if (tcp_ctask->xmstate & XMSTATE_R_HDR) { + rc = handle_xmstate_r_hdr(conn, tcp_ctask); return rc; } - if (ctask->xmstate & XMSTATE_W_HDR) { + if (tcp_ctask->xmstate & XMSTATE_W_HDR) { rc = handle_xmstate_w_hdr(conn, ctask); if (rc) return rc; } /* XXX: for data digest xmit recover */ - if (ctask->xmstate & XMSTATE_DATA_DIGEST) { + if (tcp_ctask->xmstate & XMSTATE_DATA_DIGEST) { rc = handle_xmstate_data_digest(conn, ctask); if (rc) return rc; } - if (ctask->xmstate & XMSTATE_IMM_DATA) { + if (tcp_ctask->xmstate & XMSTATE_IMM_DATA) { rc = handle_xmstate_imm_data(conn, ctask); if (rc) return rc; } - if (ctask->xmstate & XMSTATE_UNS_HDR) { + if (tcp_ctask->xmstate & XMSTATE_UNS_HDR) { BUG_ON(!ctask->unsol_count); - ctask->xmstate &= ~XMSTATE_UNS_HDR; + tcp_ctask->xmstate &= ~XMSTATE_UNS_HDR; unsolicit_head_again: rc = handle_xmstate_uns_hdr(conn, ctask); if (rc) return rc; } - if (ctask->xmstate & XMSTATE_UNS_DATA) { + if (tcp_ctask->xmstate & XMSTATE_UNS_DATA) { rc = handle_xmstate_uns_data(conn, ctask); if (rc == 1) goto unsolicit_head_again; @@ -2108,22 +1851,22 @@ unsolicit_head_again: goto done; } - if (ctask->xmstate & XMSTATE_SOL_HDR) { + if (tcp_ctask->xmstate & XMSTATE_SOL_HDR) { struct iscsi_r2t_info *r2t; - ctask->xmstate &= ~XMSTATE_SOL_HDR; - ctask->xmstate |= XMSTATE_SOL_DATA; - if (!ctask->r2t) - __kfifo_get(ctask->r2tqueue, (void*)&ctask->r2t, + tcp_ctask->xmstate &= ~XMSTATE_SOL_HDR; + tcp_ctask->xmstate |= XMSTATE_SOL_DATA; + if (!tcp_ctask->r2t) + __kfifo_get(tcp_ctask->r2tqueue, (void*)&tcp_ctask->r2t, sizeof(void*)); solicit_head_again: - r2t = ctask->r2t; + r2t = tcp_ctask->r2t; if (conn->hdrdgst_en) iscsi_hdr_digest(conn, &r2t->headbuf, (u8*)r2t->dtask->hdrext); if (iscsi_sendhdr(conn, &r2t->headbuf, r2t->data_count)) { - ctask->xmstate &= ~XMSTATE_SOL_DATA; - ctask->xmstate |= XMSTATE_SOL_HDR; + tcp_ctask->xmstate &= ~XMSTATE_SOL_DATA; + tcp_ctask->xmstate |= XMSTATE_SOL_HDR; return -EAGAIN; } @@ -2132,7 +1875,7 @@ solicit_head_again: r2t->sent); } - if (ctask->xmstate & XMSTATE_SOL_DATA) { + if (tcp_ctask->xmstate & XMSTATE_SOL_DATA) { rc = handle_xmstate_sol_data(conn, ctask); if (rc == 1) goto solicit_head_again; @@ -2145,533 +1888,122 @@ done: * Last thing to check is whether we need to send write * padding. Note that we check for xmstate equality, not just the bit. */ - if (ctask->xmstate == XMSTATE_W_PAD) + if (tcp_ctask->xmstate == XMSTATE_W_PAD) rc = handle_xmstate_w_pad(conn, ctask); return rc; } -/** - * iscsi_data_xmit - xmit any command into the scheduled connection - * @conn: iscsi connection - * - * Notes: - * The function can return -EAGAIN in which case the caller must - * re-schedule it again later or recover. '0' return code means - * successful xmit. - **/ -static int -iscsi_data_xmit(struct iscsi_conn *conn) +static struct iscsi_cls_conn * +iscsi_tcp_conn_create(struct iscsi_cls_session *cls_session, uint32_t conn_idx) { - if (unlikely(conn->suspend_tx)) { - debug_tcp("conn %d Tx suspended!\n", conn->id); - return 0; - } + struct iscsi_conn *conn; + struct iscsi_cls_conn *cls_conn; + struct iscsi_tcp_conn *tcp_conn; + cls_conn = iscsi_conn_setup(cls_session, conn_idx); + if (!cls_conn) + return NULL; + conn = cls_conn->dd_data; /* - * Transmit in the following order: - * - * 1) un-finished xmit (ctask or mtask) - * 2) immediate control PDUs - * 3) write data - * 4) SCSI commands - * 5) non-immediate control PDUs - * - * No need to lock around __kfifo_get as long as - * there's one producer and one consumer. + * due to strange issues with iser these are not set + * in iscsi_conn_setup */ + conn->max_recv_dlength = DEFAULT_MAX_RECV_DATA_SEGMENT_LENGTH; - BUG_ON(conn->ctask && conn->mtask); - - if (conn->ctask) { - if (iscsi_ctask_xmit(conn, conn->ctask)) - goto again; - /* done with this in-progress ctask */ - conn->ctask = NULL; - } - if (conn->mtask) { - if (iscsi_mtask_xmit(conn, conn->mtask)) - goto again; - /* done with this in-progress mtask */ - conn->mtask = NULL; - } - - /* process immediate first */ - if (unlikely(__kfifo_len(conn->immqueue))) { - struct iscsi_session *session = conn->session; - while (__kfifo_get(conn->immqueue, (void*)&conn->mtask, - sizeof(void*))) { - if (iscsi_mtask_xmit(conn, conn->mtask)) - goto again; - - if (conn->mtask->hdr.itt == - cpu_to_be32(ISCSI_RESERVED_TAG)) { - spin_lock_bh(&session->lock); - __kfifo_put(session->mgmtpool.queue, - (void*)&conn->mtask, sizeof(void*)); - spin_unlock_bh(&session->lock); - } - } - /* done with this mtask */ - conn->mtask = NULL; - } - - /* process write queue */ - while (__kfifo_get(conn->writequeue, (void*)&conn->ctask, - sizeof(void*))) { - if (iscsi_ctask_xmit(conn, conn->ctask)) - goto again; - } - - /* process command queue */ - while (__kfifo_get(conn->xmitqueue, (void*)&conn->ctask, - sizeof(void*))) { - if (iscsi_ctask_xmit(conn, conn->ctask)) - goto again; - } - /* done with this ctask */ - conn->ctask = NULL; - - /* process the rest control plane PDUs, if any */ - if (unlikely(__kfifo_len(conn->mgmtqueue))) { - struct iscsi_session *session = conn->session; - - while (__kfifo_get(conn->mgmtqueue, (void*)&conn->mtask, - sizeof(void*))) { - if (iscsi_mtask_xmit(conn, conn->mtask)) - goto again; - - if (conn->mtask->hdr.itt == - cpu_to_be32(ISCSI_RESERVED_TAG)) { - spin_lock_bh(&session->lock); - __kfifo_put(session->mgmtpool.queue, - (void*)&conn->mtask, - sizeof(void*)); - spin_unlock_bh(&session->lock); - } - } - /* done with this mtask */ - conn->mtask = NULL; - } + tcp_conn = kzalloc(sizeof(*tcp_conn), GFP_KERNEL); + if (!tcp_conn) + goto tcp_conn_alloc_fail; + + conn->dd_data = tcp_conn; + tcp_conn->iscsi_conn = conn; + tcp_conn->in_progress = IN_PROGRESS_WAIT_HEADER; + /* initial operational parameters */ + tcp_conn->hdr_size = sizeof(struct iscsi_hdr); + tcp_conn->data_size = DEFAULT_MAX_RECV_DATA_SEGMENT_LENGTH; - return 0; + /* allocate initial PDU receive place holder */ + if (tcp_conn->data_size <= PAGE_SIZE) + tcp_conn->data = kmalloc(tcp_conn->data_size, GFP_KERNEL); + else + tcp_conn->data = (void*)__get_free_pages(GFP_KERNEL, + get_order(tcp_conn->data_size)); + if (!tcp_conn->data) + goto max_recv_dlenght_alloc_fail; -again: - if (unlikely(conn->suspend_tx)) - return 0; + return cls_conn; - return -EAGAIN; +max_recv_dlenght_alloc_fail: + kfree(tcp_conn); +tcp_conn_alloc_fail: + iscsi_conn_teardown(cls_conn); + return NULL; } static void -iscsi_xmitworker(void *data) +iscsi_tcp_conn_destroy(struct iscsi_cls_conn *cls_conn) { - struct iscsi_conn *conn = data; - - /* - * serialize Xmit worker on a per-connection basis. - */ - mutex_lock(&conn->xmitmutex); - if (iscsi_data_xmit(conn)) - scsi_queue_work(conn->session->host, &conn->xmitwork); - mutex_unlock(&conn->xmitmutex); -} + struct iscsi_conn *conn = cls_conn->dd_data; + struct iscsi_tcp_conn *tcp_conn = conn->dd_data; + int digest = 0; -#define FAILURE_BAD_HOST 1 -#define FAILURE_SESSION_FAILED 2 -#define FAILURE_SESSION_FREED 3 -#define FAILURE_WINDOW_CLOSED 4 -#define FAILURE_SESSION_TERMINATE 5 + if (conn->hdrdgst_en || conn->datadgst_en) + digest = 1; -static int -iscsi_queuecommand(struct scsi_cmnd *sc, void (*done)(struct scsi_cmnd *)) -{ - struct Scsi_Host *host; - int reason = 0; - struct iscsi_session *session; - struct iscsi_conn *conn = NULL; - struct iscsi_cmd_task *ctask = NULL; + iscsi_conn_teardown(cls_conn); - sc->scsi_done = done; - sc->result = 0; + /* now free tcp_conn */ + if (digest) { + if (tcp_conn->tx_tfm) + crypto_free_tfm(tcp_conn->tx_tfm); + if (tcp_conn->rx_tfm) + crypto_free_tfm(tcp_conn->rx_tfm); + if (tcp_conn->data_tx_tfm) + crypto_free_tfm(tcp_conn->data_tx_tfm); + if (tcp_conn->data_rx_tfm) + crypto_free_tfm(tcp_conn->data_rx_tfm); + } - host = sc->device->host; - session = iscsi_hostdata(host->hostdata); - BUG_ON(host != session->host); + /* free conn->data, size = MaxRecvDataSegmentLength */ + if (tcp_conn->data_size <= PAGE_SIZE) + kfree(tcp_conn->data); + else + free_pages((unsigned long)tcp_conn->data, + get_order(tcp_conn->data_size)); + kfree(tcp_conn); +} - spin_lock(&session->lock); +static int +iscsi_tcp_conn_bind(struct iscsi_cls_session *cls_session, + struct iscsi_cls_conn *cls_conn, uint32_t transport_fd, + int is_leading) +{ + struct iscsi_conn *conn = cls_conn->dd_data; + struct iscsi_tcp_conn *tcp_conn = conn->dd_data; + struct sock *sk; + struct socket *sock; + int err; - if (session->state != ISCSI_STATE_LOGGED_IN) { - if (session->state == ISCSI_STATE_FAILED) { - reason = FAILURE_SESSION_FAILED; - goto reject; - } else if (session->state == ISCSI_STATE_TERMINATE) { - reason = FAILURE_SESSION_TERMINATE; - goto fault; - } - reason = FAILURE_SESSION_FREED; - goto fault; + /* lookup for existing socket */ + sock = sockfd_lookup(transport_fd, &err); + if (!sock) { + printk(KERN_ERR "iscsi_tcp: sockfd_lookup failed %d\n", err); + return -EEXIST; } - /* - * Check for iSCSI window and take care of CmdSN wrap-around - */ - if ((int)(session->max_cmdsn - session->cmdsn) < 0) { - reason = FAILURE_WINDOW_CLOSED; - goto reject; - } + err = iscsi_conn_bind(cls_session, cls_conn, is_leading); + if (err) + return err; - conn = session->leadconn; + if (conn->stop_stage != STOP_CONN_SUSPEND) { + /* bind iSCSI connection and socket */ + tcp_conn->sock = sock; - __kfifo_get(session->cmdpool.queue, (void*)&ctask, sizeof(void*)); - BUG_ON(ctask->sc); - - sc->SCp.phase = session->age; - sc->SCp.ptr = (char*)ctask; - iscsi_cmd_init(conn, ctask, sc); - - __kfifo_put(conn->xmitqueue, (void*)&ctask, sizeof(void*)); - debug_scsi( - "ctask enq [%s cid %d sc %lx itt 0x%x len %d cmdsn %d win %d]\n", - sc->sc_data_direction == DMA_TO_DEVICE ? "write" : "read", - conn->id, (long)sc, ctask->itt, sc->request_bufflen, - session->cmdsn, session->max_cmdsn - session->exp_cmdsn + 1); - spin_unlock(&session->lock); - - scsi_queue_work(host, &conn->xmitwork); - return 0; - -reject: - spin_unlock(&session->lock); - debug_scsi("cmd 0x%x rejected (%d)\n", sc->cmnd[0], reason); - return SCSI_MLQUEUE_HOST_BUSY; - -fault: - spin_unlock(&session->lock); - printk(KERN_ERR "iscsi_tcp: cmd 0x%x is not queued (%d)\n", - sc->cmnd[0], reason); - sc->sense_buffer[0] = 0x70; - sc->sense_buffer[2] = NOT_READY; - sc->sense_buffer[7] = 0x6; - sc->sense_buffer[12] = 0x08; - sc->sense_buffer[13] = 0x00; - sc->result = (DID_NO_CONNECT << 16); - sc->resid = sc->request_bufflen; - sc->scsi_done(sc); - return 0; -} - -static int -iscsi_change_queue_depth(struct scsi_device *sdev, int depth) -{ - if (depth > ISCSI_MAX_CMD_PER_LUN) - depth = ISCSI_MAX_CMD_PER_LUN; - scsi_adjust_queue_depth(sdev, scsi_get_tag_type(sdev), depth); - return sdev->queue_depth; -} - -static int -iscsi_pool_init(struct iscsi_queue *q, int max, void ***items, int item_size) -{ - int i; - - *items = kmalloc(max * sizeof(void*), GFP_KERNEL); - if (*items == NULL) - return -ENOMEM; - - q->max = max; - q->pool = kmalloc(max * sizeof(void*), GFP_KERNEL); - if (q->pool == NULL) { - kfree(*items); - return -ENOMEM; - } - - q->queue = kfifo_init((void*)q->pool, max * sizeof(void*), - GFP_KERNEL, NULL); - if (q->queue == ERR_PTR(-ENOMEM)) { - kfree(q->pool); - kfree(*items); - return -ENOMEM; - } - - for (i = 0; i < max; i++) { - q->pool[i] = kmalloc(item_size, GFP_KERNEL); - if (q->pool[i] == NULL) { - int j; - - for (j = 0; j < i; j++) - kfree(q->pool[j]); - - kfifo_free(q->queue); - kfree(q->pool); - kfree(*items); - return -ENOMEM; - } - memset(q->pool[i], 0, item_size); - (*items)[i] = q->pool[i]; - __kfifo_put(q->queue, (void*)&q->pool[i], sizeof(void*)); - } - return 0; -} - -static void -iscsi_pool_free(struct iscsi_queue *q, void **items) -{ - int i; - - for (i = 0; i < q->max; i++) - kfree(items[i]); - kfree(q->pool); - kfree(items); -} - -static struct iscsi_cls_conn * -iscsi_conn_create(struct iscsi_cls_session *cls_session, uint32_t conn_idx) -{ - struct Scsi_Host *shost = iscsi_session_to_shost(cls_session); - struct iscsi_session *session = iscsi_hostdata(shost->hostdata); - struct iscsi_conn *conn; - struct iscsi_cls_conn *cls_conn; - - cls_conn = iscsi_create_conn(cls_session, conn_idx); - if (!cls_conn) - return NULL; - conn = cls_conn->dd_data; - memset(conn, 0, sizeof(*conn)); - - conn->cls_conn = cls_conn; - conn->c_stage = ISCSI_CONN_INITIAL_STAGE; - conn->in_progress = IN_PROGRESS_WAIT_HEADER; - conn->id = conn_idx; - conn->exp_statsn = 0; - conn->tmabort_state = TMABORT_INITIAL; - - /* initial operational parameters */ - conn->hdr_size = sizeof(struct iscsi_hdr); - conn->data_size = DEFAULT_MAX_RECV_DATA_SEGMENT_LENGTH; - conn->max_recv_dlength = DEFAULT_MAX_RECV_DATA_SEGMENT_LENGTH; - - /* initialize general xmit PDU commands queue */ - conn->xmitqueue = kfifo_alloc(session->cmds_max * sizeof(void*), - GFP_KERNEL, NULL); - if (conn->xmitqueue == ERR_PTR(-ENOMEM)) - goto xmitqueue_alloc_fail; - - /* initialize write response PDU commands queue */ - conn->writequeue = kfifo_alloc(session->cmds_max * sizeof(void*), - GFP_KERNEL, NULL); - if (conn->writequeue == ERR_PTR(-ENOMEM)) - goto writequeue_alloc_fail; - - /* initialize general immediate & non-immediate PDU commands queue */ - conn->immqueue = kfifo_alloc(session->mgmtpool_max * sizeof(void*), - GFP_KERNEL, NULL); - if (conn->immqueue == ERR_PTR(-ENOMEM)) - goto immqueue_alloc_fail; - - conn->mgmtqueue = kfifo_alloc(session->mgmtpool_max * sizeof(void*), - GFP_KERNEL, NULL); - if (conn->mgmtqueue == ERR_PTR(-ENOMEM)) - goto mgmtqueue_alloc_fail; - - INIT_WORK(&conn->xmitwork, iscsi_xmitworker, conn); - - /* allocate login_mtask used for the login/text sequences */ - spin_lock_bh(&session->lock); - if (!__kfifo_get(session->mgmtpool.queue, - (void*)&conn->login_mtask, - sizeof(void*))) { - spin_unlock_bh(&session->lock); - goto login_mtask_alloc_fail; - } - spin_unlock_bh(&session->lock); - - /* allocate initial PDU receive place holder */ - if (conn->data_size <= PAGE_SIZE) - conn->data = kmalloc(conn->data_size, GFP_KERNEL); - else - conn->data = (void*)__get_free_pages(GFP_KERNEL, - get_order(conn->data_size)); - if (!conn->data) - goto max_recv_dlenght_alloc_fail; - - init_timer(&conn->tmabort_timer); - mutex_init(&conn->xmitmutex); - init_waitqueue_head(&conn->ehwait); - - return cls_conn; - -max_recv_dlenght_alloc_fail: - spin_lock_bh(&session->lock); - __kfifo_put(session->mgmtpool.queue, (void*)&conn->login_mtask, - sizeof(void*)); - spin_unlock_bh(&session->lock); -login_mtask_alloc_fail: - kfifo_free(conn->mgmtqueue); -mgmtqueue_alloc_fail: - kfifo_free(conn->immqueue); -immqueue_alloc_fail: - kfifo_free(conn->writequeue); -writequeue_alloc_fail: - kfifo_free(conn->xmitqueue); -xmitqueue_alloc_fail: - iscsi_destroy_conn(cls_conn); - return NULL; -} - -static void -iscsi_conn_destroy(struct iscsi_cls_conn *cls_conn) -{ - struct iscsi_conn *conn = cls_conn->dd_data; - struct iscsi_session *session = conn->session; - unsigned long flags; - - mutex_lock(&conn->xmitmutex); - set_bit(SUSPEND_BIT, &conn->suspend_tx); - if (conn->c_stage == ISCSI_CONN_INITIAL_STAGE && conn->sock) { - struct sock *sk = conn->sock->sk; - - /* - * conn_start() has never been called! - * need to cleanup the socket. - */ - write_lock_bh(&sk->sk_callback_lock); - set_bit(SUSPEND_BIT, &conn->suspend_rx); - write_unlock_bh(&sk->sk_callback_lock); - - sock_hold(conn->sock->sk); - iscsi_conn_restore_callbacks(conn); - sock_put(conn->sock->sk); - sock_release(conn->sock); - conn->sock = NULL; - } - - spin_lock_bh(&session->lock); - conn->c_stage = ISCSI_CONN_CLEANUP_WAIT; - if (session->leadconn == conn) { - /* - * leading connection? then give up on recovery. - */ - session->state = ISCSI_STATE_TERMINATE; - wake_up(&conn->ehwait); - } - spin_unlock_bh(&session->lock); - - mutex_unlock(&conn->xmitmutex); - - /* - * Block until all in-progress commands for this connection - * time out or fail. - */ - for (;;) { - spin_lock_irqsave(session->host->host_lock, flags); - if (!session->host->host_busy) { /* OK for ERL == 0 */ - spin_unlock_irqrestore(session->host->host_lock, flags); - break; - } - spin_unlock_irqrestore(session->host->host_lock, flags); - msleep_interruptible(500); - printk("conn_destroy(): host_busy %d host_failed %d\n", - session->host->host_busy, session->host->host_failed); - /* - * force eh_abort() to unblock - */ - wake_up(&conn->ehwait); - } - - /* now free crypto */ - if (conn->hdrdgst_en || conn->datadgst_en) { - if (conn->tx_tfm) - crypto_free_tfm(conn->tx_tfm); - if (conn->rx_tfm) - crypto_free_tfm(conn->rx_tfm); - if (conn->data_tx_tfm) - crypto_free_tfm(conn->data_tx_tfm); - if (conn->data_rx_tfm) - crypto_free_tfm(conn->data_rx_tfm); - } - - /* free conn->data, size = MaxRecvDataSegmentLength */ - if (conn->data_size <= PAGE_SIZE) - kfree(conn->data); - else - free_pages((unsigned long)conn->data, - get_order(conn->data_size)); - - spin_lock_bh(&session->lock); - __kfifo_put(session->mgmtpool.queue, (void*)&conn->login_mtask, - sizeof(void*)); - list_del(&conn->item); - if (list_empty(&session->connections)) - session->leadconn = NULL; - if (session->leadconn && session->leadconn == conn) - session->leadconn = container_of(session->connections.next, - struct iscsi_conn, item); - - if (session->leadconn == NULL) - /* none connections exits.. reset sequencing */ - session->cmdsn = session->max_cmdsn = session->exp_cmdsn = 1; - spin_unlock_bh(&session->lock); - - kfifo_free(conn->xmitqueue); - kfifo_free(conn->writequeue); - kfifo_free(conn->immqueue); - kfifo_free(conn->mgmtqueue); - - iscsi_destroy_conn(cls_conn); -} - -static int -iscsi_conn_bind(struct iscsi_cls_session *cls_session, - struct iscsi_cls_conn *cls_conn, uint32_t transport_fd, - int is_leading) -{ - struct Scsi_Host *shost = iscsi_session_to_shost(cls_session); - struct iscsi_session *session = iscsi_hostdata(shost->hostdata); - struct iscsi_conn *tmp = ERR_PTR(-EEXIST), *conn = cls_conn->dd_data; - struct sock *sk; - struct socket *sock; - int err; - - /* lookup for existing socket */ - sock = sockfd_lookup(transport_fd, &err); - if (!sock) { - printk(KERN_ERR "iscsi_tcp: sockfd_lookup failed %d\n", err); - return -EEXIST; - } - - /* lookup for existing connection */ - spin_lock_bh(&session->lock); - list_for_each_entry(tmp, &session->connections, item) { - if (tmp == conn) { - if (conn->c_stage != ISCSI_CONN_STOPPED || - conn->stop_stage == STOP_CONN_TERM) { - printk(KERN_ERR "iscsi_tcp: can't bind " - "non-stopped connection (%d:%d)\n", - conn->c_stage, conn->stop_stage); - spin_unlock_bh(&session->lock); - return -EIO; - } - break; - } - } - if (tmp != conn) { - /* bind new iSCSI connection to session */ - conn->session = session; - - list_add(&conn->item, &session->connections); - } - spin_unlock_bh(&session->lock); - - if (conn->stop_stage != STOP_CONN_SUSPEND) { - /* bind iSCSI connection and socket */ - conn->sock = sock; - - /* setup Socket parameters */ - sk = sock->sk; - sk->sk_reuse = 1; - sk->sk_sndtimeo = 15 * HZ; /* FIXME: make it configurable */ - sk->sk_allocation = GFP_ATOMIC; + /* setup Socket parameters */ + sk = sock->sk; + sk->sk_reuse = 1; + sk->sk_sndtimeo = 15 * HZ; /* FIXME: make it configurable */ + sk->sk_allocation = GFP_ATOMIC; /* FIXME: disable Nagle's algorithm */ @@ -2679,488 +2011,78 @@ iscsi_conn_bind(struct iscsi_cls_session * Intercept TCP callbacks for sendfile like receive * processing. */ + conn->recv_lock = &sk->sk_callback_lock; iscsi_conn_set_callbacks(conn); - - conn->sendpage = conn->sock->ops->sendpage; - + tcp_conn->sendpage = tcp_conn->sock->ops->sendpage; /* * set receive state machine into initial state */ - conn->in_progress = IN_PROGRESS_WAIT_HEADER; + tcp_conn->in_progress = IN_PROGRESS_WAIT_HEADER; } - if (is_leading) - session->leadconn = conn; - - /* - * Unblock xmitworker(), Login Phase will pass through. - */ - clear_bit(SUSPEND_BIT, &conn->suspend_rx); - clear_bit(SUSPEND_BIT, &conn->suspend_tx); - return 0; } -static int -iscsi_conn_start(struct iscsi_cls_conn *cls_conn) +static void +iscsi_tcp_cleanup_ctask(struct iscsi_conn *conn, struct iscsi_cmd_task *ctask) { - struct iscsi_conn *conn = cls_conn->dd_data; - struct iscsi_session *session = conn->session; - struct sock *sk; - - /* FF phase warming up... */ - - if (session == NULL) { - printk(KERN_ERR "iscsi_tcp: can't start unbound connection\n"); - return -EPERM; - } - - sk = conn->sock->sk; + struct iscsi_tcp_cmd_task *tcp_ctask = ctask->dd_data; + struct iscsi_r2t_info *r2t; - write_lock_bh(&sk->sk_callback_lock); - spin_lock_bh(&session->lock); - conn->c_stage = ISCSI_CONN_STARTED; - session->state = ISCSI_STATE_LOGGED_IN; + /* flush ctask's r2t queues */ + while (__kfifo_get(tcp_ctask->r2tqueue, (void*)&r2t, sizeof(void*))) + __kfifo_put(tcp_ctask->r2tpool.queue, (void*)&r2t, + sizeof(void*)); - switch(conn->stop_stage) { - case STOP_CONN_RECOVER: - /* - * unblock eh_abort() if it is blocked. re-try all - * commands after successful recovery - */ - session->conn_cnt++; - conn->stop_stage = 0; - conn->tmabort_state = TMABORT_INITIAL; - session->age++; - wake_up(&conn->ehwait); - break; - case STOP_CONN_TERM: - session->conn_cnt++; - conn->stop_stage = 0; - break; - case STOP_CONN_SUSPEND: - conn->stop_stage = 0; - clear_bit(SUSPEND_BIT, &conn->suspend_rx); - clear_bit(SUSPEND_BIT, &conn->suspend_tx); - break; - default: - break; - } - spin_unlock_bh(&session->lock); - write_unlock_bh(&sk->sk_callback_lock); - - return 0; + __iscsi_ctask_cleanup(conn, ctask); } static void -iscsi_conn_stop(struct iscsi_cls_conn *cls_conn, int flag) +iscsi_tcp_suspend_conn_rx(struct iscsi_conn *conn) { - struct iscsi_conn *conn = cls_conn->dd_data; - struct iscsi_session *session = conn->session; + struct iscsi_tcp_conn *tcp_conn = conn->dd_data; struct sock *sk; - unsigned long flags; - BUG_ON(!conn->sock); - sk = conn->sock->sk; + if (!tcp_conn->sock) + return; + + sk = tcp_conn->sock->sk; write_lock_bh(&sk->sk_callback_lock); - set_bit(SUSPEND_BIT, &conn->suspend_rx); + set_bit(ISCSI_SUSPEND_BIT, &conn->suspend_rx); write_unlock_bh(&sk->sk_callback_lock); - - mutex_lock(&conn->xmitmutex); - - spin_lock_irqsave(session->host->host_lock, flags); - spin_lock(&session->lock); - conn->stop_stage = flag; - conn->c_stage = ISCSI_CONN_STOPPED; - set_bit(SUSPEND_BIT, &conn->suspend_tx); - - if (flag != STOP_CONN_SUSPEND) - session->conn_cnt--; - - if (session->conn_cnt == 0 || session->leadconn == conn) - session->state = ISCSI_STATE_FAILED; - - spin_unlock(&session->lock); - spin_unlock_irqrestore(session->host->host_lock, flags); - - if (flag == STOP_CONN_TERM || flag == STOP_CONN_RECOVER) { - struct iscsi_cmd_task *ctask; - struct iscsi_mgmt_task *mtask; - - /* - * Socket must go now. - */ - sock_hold(conn->sock->sk); - iscsi_conn_restore_callbacks(conn); - sock_put(conn->sock->sk); - - /* - * flush xmit queues. - */ - spin_lock_bh(&session->lock); - while (__kfifo_get(conn->writequeue, (void*)&ctask, - sizeof(void*)) || - __kfifo_get(conn->xmitqueue, (void*)&ctask, - sizeof(void*))) { - struct iscsi_r2t_info *r2t; - - /* - * flush ctask's r2t queues - */ - while (__kfifo_get(ctask->r2tqueue, (void*)&r2t, - sizeof(void*))) - __kfifo_put(ctask->r2tpool.queue, (void*)&r2t, - sizeof(void*)); - - spin_unlock_bh(&session->lock); - local_bh_disable(); - iscsi_ctask_cleanup(conn, ctask); - local_bh_enable(); - spin_lock_bh(&session->lock); - } - conn->ctask = NULL; - while (__kfifo_get(conn->immqueue, (void*)&mtask, - sizeof(void*)) || - __kfifo_get(conn->mgmtqueue, (void*)&mtask, - sizeof(void*))) { - __kfifo_put(session->mgmtpool.queue, - (void*)&mtask, sizeof(void*)); - } - conn->mtask = NULL; - spin_unlock_bh(&session->lock); - - /* - * release socket only after we stopped data_xmit() - * activity and flushed all outstandings - */ - sock_release(conn->sock); - conn->sock = NULL; - - /* - * for connection level recovery we should not calculate - * header digest. conn->hdr_size used for optimization - * in hdr_extract() and will be re-negotiated at - * set_param() time. - */ - if (flag == STOP_CONN_RECOVER) { - conn->hdr_size = sizeof(struct iscsi_hdr); - conn->hdrdgst_en = 0; - conn->datadgst_en = 0; - } - } - mutex_unlock(&conn->xmitmutex); } -static int -iscsi_conn_send_generic(struct iscsi_conn *conn, struct iscsi_hdr *hdr, - char *data, uint32_t data_size) -{ - struct iscsi_session *session = conn->session; - struct iscsi_nopout *nop = (struct iscsi_nopout *)hdr; - struct iscsi_mgmt_task *mtask; - - spin_lock_bh(&session->lock); - if (session->state == ISCSI_STATE_TERMINATE) { - spin_unlock_bh(&session->lock); - return -EPERM; - } - if (hdr->opcode == (ISCSI_OP_LOGIN | ISCSI_OP_IMMEDIATE) || - hdr->opcode == (ISCSI_OP_TEXT | ISCSI_OP_IMMEDIATE)) - /* - * Login and Text are sent serially, in - * request-followed-by-response sequence. - * Same mtask can be used. Same ITT must be used. - * Note that login_mtask is preallocated at conn_create(). - */ - mtask = conn->login_mtask; - else { - BUG_ON(conn->c_stage == ISCSI_CONN_INITIAL_STAGE); - BUG_ON(conn->c_stage == ISCSI_CONN_STOPPED); - - if (!__kfifo_get(session->mgmtpool.queue, - (void*)&mtask, sizeof(void*))) { - spin_unlock_bh(&session->lock); - return -ENOSPC; - } - } - - /* - * pre-format CmdSN and ExpStatSN for outgoing PDU. - */ - if (hdr->itt != cpu_to_be32(ISCSI_RESERVED_TAG)) { - hdr->itt = mtask->itt | (conn->id << CID_SHIFT) | - (session->age << AGE_SHIFT); - nop->cmdsn = cpu_to_be32(session->cmdsn); - if (conn->c_stage == ISCSI_CONN_STARTED && - !(hdr->opcode & ISCSI_OP_IMMEDIATE)) - session->cmdsn++; - } else - /* do not advance CmdSN */ - nop->cmdsn = cpu_to_be32(session->cmdsn); - - nop->exp_statsn = cpu_to_be32(conn->exp_statsn); - - memcpy(&mtask->hdr, hdr, sizeof(struct iscsi_hdr)); - - iscsi_buf_init_virt(&mtask->headbuf, (char*)&mtask->hdr, - sizeof(struct iscsi_hdr)); - - spin_unlock_bh(&session->lock); - - if (data_size) { - memcpy(mtask->data, data, data_size); - mtask->data_count = data_size; - } else - mtask->data_count = 0; - - mtask->xmstate = XMSTATE_IMM_HDR; - - if (mtask->data_count) { - iscsi_buf_init_iov(&mtask->sendbuf, (char*)mtask->data, - mtask->data_count); - } - - debug_scsi("mgmtpdu [op 0x%x hdr->itt 0x%x datalen %d]\n", - hdr->opcode, hdr->itt, data_size); - - /* - * since send_pdu() could be called at least from two contexts, - * we need to serialize __kfifo_put, so we don't have to take - * additional lock on fast data-path - */ - if (hdr->opcode & ISCSI_OP_IMMEDIATE) - __kfifo_put(conn->immqueue, (void*)&mtask, sizeof(void*)); - else - __kfifo_put(conn->mgmtqueue, (void*)&mtask, sizeof(void*)); - - scsi_queue_work(session->host, &conn->xmitwork); - return 0; -} - -static int -iscsi_eh_host_reset(struct scsi_cmnd *sc) +static void +iscsi_tcp_terminate_conn(struct iscsi_conn *conn) { - struct iscsi_cmd_task *ctask = (struct iscsi_cmd_task *)sc->SCp.ptr; - struct iscsi_conn *conn = ctask->conn; - struct iscsi_session *session = conn->session; + struct iscsi_tcp_conn *tcp_conn = conn->dd_data; - spin_lock_bh(&session->lock); - if (session->state == ISCSI_STATE_TERMINATE) { - debug_scsi("failing host reset: session terminated " - "[CID %d age %d]", conn->id, session->age); - spin_unlock_bh(&session->lock); - return FAILED; - } - spin_unlock_bh(&session->lock); + if (!tcp_conn->sock) + return; - debug_scsi("failing connection CID %d due to SCSI host reset " - "[itt 0x%x age %d]", conn->id, ctask->itt, - session->age); - iscsi_conn_failure(conn, ISCSI_ERR_CONN_FAILED); + sock_hold(tcp_conn->sock->sk); + iscsi_conn_restore_callbacks(conn); + sock_put(tcp_conn->sock->sk); - return SUCCESS; + sock_release(tcp_conn->sock); + tcp_conn->sock = NULL; + conn->recv_lock = NULL; } +/* called with host lock */ static void -iscsi_tmabort_timedout(unsigned long data) -{ - struct iscsi_cmd_task *ctask = (struct iscsi_cmd_task *)data; - struct iscsi_conn *conn = ctask->conn; - struct iscsi_session *session = conn->session; - - spin_lock(&session->lock); - if (conn->tmabort_state == TMABORT_INITIAL) { - __kfifo_put(session->mgmtpool.queue, - (void*)&ctask->mtask, sizeof(void*)); - conn->tmabort_state = TMABORT_TIMEDOUT; - debug_scsi("tmabort timedout [sc %lx itt 0x%x]\n", - (long)ctask->sc, ctask->itt); - /* unblock eh_abort() */ - wake_up(&conn->ehwait); - } - spin_unlock(&session->lock); -} - -static int -iscsi_eh_abort(struct scsi_cmnd *sc) +iscsi_tcp_mgmt_init(struct iscsi_conn *conn, struct iscsi_mgmt_task *mtask, + char *data, uint32_t data_size) { - int rc; - struct iscsi_cmd_task *ctask = (struct iscsi_cmd_task *)sc->SCp.ptr; - struct iscsi_conn *conn = ctask->conn; - struct iscsi_session *session = conn->session; - - conn->eh_abort_cnt++; - debug_scsi("aborting [sc %lx itt 0x%x]\n", (long)sc, ctask->itt); - - /* - * two cases for ERL=0 here: - * - * 1) connection-level failure; - * 2) recovery due protocol error; - */ - mutex_lock(&conn->xmitmutex); - spin_lock_bh(&session->lock); - if (session->state != ISCSI_STATE_LOGGED_IN) { - if (session->state == ISCSI_STATE_TERMINATE) { - spin_unlock_bh(&session->lock); - mutex_unlock(&conn->xmitmutex); - goto failed; - } - spin_unlock_bh(&session->lock); - } else { - struct iscsi_tm *hdr = &conn->tmhdr; - - /* - * Still LOGGED_IN... - */ - - if (!ctask->sc || sc->SCp.phase != session->age) { - /* - * 1) ctask completed before time out. But session - * is still ok => Happy Retry. - * 2) session was re-open during time out of ctask. - */ - spin_unlock_bh(&session->lock); - mutex_unlock(&conn->xmitmutex); - goto success; - } - conn->tmabort_state = TMABORT_INITIAL; - spin_unlock_bh(&session->lock); - - /* - * ctask timed out but session is OK - * ERL=0 requires task mgmt abort to be issued on each - * failed command. requests must be serialized. - */ - memset(hdr, 0, sizeof(struct iscsi_tm)); - hdr->opcode = ISCSI_OP_SCSI_TMFUNC | ISCSI_OP_IMMEDIATE; - hdr->flags = ISCSI_TM_FUNC_ABORT_TASK; - hdr->flags |= ISCSI_FLAG_CMD_FINAL; - memcpy(hdr->lun, ctask->hdr.lun, sizeof(hdr->lun)); - hdr->rtt = ctask->hdr.itt; - hdr->refcmdsn = ctask->hdr.cmdsn; + struct iscsi_tcp_mgmt_task *tcp_mtask = mtask->dd_data; - rc = iscsi_conn_send_generic(conn, (struct iscsi_hdr *)hdr, - NULL, 0); - if (rc) { - iscsi_conn_failure(conn, ISCSI_ERR_CONN_FAILED); - debug_scsi("abort sent failure [itt 0x%x]", ctask->itt); - } else { - struct iscsi_r2t_info *r2t; - - /* - * TMF abort vs. TMF response race logic - */ - spin_lock_bh(&session->lock); - ctask->mtask = (struct iscsi_mgmt_task *) - session->mgmt_cmds[(hdr->itt & ITT_MASK) - - ISCSI_MGMT_ITT_OFFSET]; - /* - * have to flush r2tqueue to avoid r2t leaks - */ - while (__kfifo_get(ctask->r2tqueue, (void*)&r2t, - sizeof(void*))) { - __kfifo_put(ctask->r2tpool.queue, (void*)&r2t, - sizeof(void*)); - } - if (conn->tmabort_state == TMABORT_INITIAL) { - conn->tmfcmd_pdus_cnt++; - conn->tmabort_timer.expires = 3*HZ + jiffies; - conn->tmabort_timer.function = - iscsi_tmabort_timedout; - conn->tmabort_timer.data = (unsigned long)ctask; - add_timer(&conn->tmabort_timer); - debug_scsi("abort sent [itt 0x%x]", ctask->itt); - } else { - if (!ctask->sc || - conn->tmabort_state == TMABORT_SUCCESS) { - conn->tmabort_state = TMABORT_INITIAL; - spin_unlock_bh(&session->lock); - mutex_unlock(&conn->xmitmutex); - goto success; - } - conn->tmabort_state = TMABORT_INITIAL; - iscsi_conn_failure(conn, ISCSI_ERR_CONN_FAILED); - } - spin_unlock_bh(&session->lock); - } - } - mutex_unlock(&conn->xmitmutex); - - - /* - * block eh thread until: - * - * 1) abort response; - * 2) abort timeout; - * 3) session re-opened; - * 4) session terminated; - */ - for (;;) { - int p_state = session->state; - - rc = wait_event_interruptible(conn->ehwait, - (p_state == ISCSI_STATE_LOGGED_IN ? - (session->state == ISCSI_STATE_TERMINATE || - conn->tmabort_state != TMABORT_INITIAL) : - (session->state == ISCSI_STATE_TERMINATE || - session->state == ISCSI_STATE_LOGGED_IN))); - if (rc) { - /* shutdown.. */ - session->state = ISCSI_STATE_TERMINATE; - goto failed; - } - - if (signal_pending(current)) - flush_signals(current); - - if (session->state == ISCSI_STATE_TERMINATE) - goto failed; - - spin_lock_bh(&session->lock); - if (sc->SCp.phase == session->age && - (conn->tmabort_state == TMABORT_TIMEDOUT || - conn->tmabort_state == TMABORT_FAILED)) { - conn->tmabort_state = TMABORT_INITIAL; - if (!ctask->sc) { - /* - * ctask completed before tmf abort response or - * time out. - * But session is still ok => Happy Retry. - */ - spin_unlock_bh(&session->lock); - break; - } - spin_unlock_bh(&session->lock); - iscsi_conn_failure(conn, ISCSI_ERR_CONN_FAILED); - continue; - } - spin_unlock_bh(&session->lock); - break; - } - -success: - debug_scsi("abort success [sc %lx itt 0x%x]\n", (long)sc, ctask->itt); - rc = SUCCESS; - goto exit; - -failed: - debug_scsi("abort failed [sc %lx itt 0x%x]\n", (long)sc, ctask->itt); - rc = FAILED; - -exit: - del_timer_sync(&conn->tmabort_timer); - - mutex_lock(&conn->xmitmutex); - if (conn->sock) { - struct sock *sk = conn->sock->sk; + iscsi_buf_init_virt(&tcp_mtask->headbuf, (char*)mtask->hdr, + sizeof(struct iscsi_hdr)); + tcp_mtask->xmstate = XMSTATE_IMM_HDR; - write_lock_bh(&sk->sk_callback_lock); - iscsi_ctask_cleanup(conn, ctask); - write_unlock_bh(&sk->sk_callback_lock); - } - mutex_unlock(&conn->xmitmutex); - return rc; + if (mtask->data_count) + iscsi_buf_init_iov(&tcp_mtask->sendbuf, (char*)mtask->data, + mtask->data_count); } static int @@ -3174,6 +2096,7 @@ iscsi_r2tpool_alloc(struct iscsi_session */ for (cmd_i = 0; cmd_i < session->cmds_max; cmd_i++) { struct iscsi_cmd_task *ctask = session->cmds[cmd_i]; + struct iscsi_tcp_cmd_task *tcp_ctask = ctask->dd_data; /* * pre-allocated x4 as much r2ts to handle race when @@ -3182,16 +2105,18 @@ iscsi_r2tpool_alloc(struct iscsi_session */ /* R2T pool */ - if (iscsi_pool_init(&ctask->r2tpool, session->max_r2t * 4, - (void***)&ctask->r2ts, sizeof(struct iscsi_r2t_info))) { + if (iscsi_pool_init(&tcp_ctask->r2tpool, session->max_r2t * 4, + (void***)&tcp_ctask->r2ts, + sizeof(struct iscsi_r2t_info))) { goto r2t_alloc_fail; } /* R2T xmit queue */ - ctask->r2tqueue = kfifo_alloc( + tcp_ctask->r2tqueue = kfifo_alloc( session->max_r2t * 4 * sizeof(void*), GFP_KERNEL, NULL); - if (ctask->r2tqueue == ERR_PTR(-ENOMEM)) { - iscsi_pool_free(&ctask->r2tpool, (void**)ctask->r2ts); + if (tcp_ctask->r2tqueue == ERR_PTR(-ENOMEM)) { + iscsi_pool_free(&tcp_ctask->r2tpool, + (void**)tcp_ctask->r2ts); goto r2t_alloc_fail; } @@ -3200,24 +2125,28 @@ iscsi_r2tpool_alloc(struct iscsi_session * Data-Out PDU's within R2T-sequence can be quite big; * using mempool */ - ctask->datapool = mempool_create_slab_pool(ISCSI_DTASK_DEFAULT_MAX, - taskcache); - if (ctask->datapool == NULL) { - kfifo_free(ctask->r2tqueue); - iscsi_pool_free(&ctask->r2tpool, (void**)ctask->r2ts); + tcp_ctask->datapool = mempool_create_slab_pool(ISCSI_DTASK_DEFAULT_MAX, + taskcache); + if (tcp_ctask->datapool == NULL) { + kfifo_free(tcp_ctask->r2tqueue); + iscsi_pool_free(&tcp_ctask->r2tpool, + (void**)tcp_ctask->r2ts); goto r2t_alloc_fail; } - INIT_LIST_HEAD(&ctask->dataqueue); + INIT_LIST_HEAD(&tcp_ctask->dataqueue); } return 0; r2t_alloc_fail: for (i = 0; i < cmd_i; i++) { - mempool_destroy(session->cmds[i]->datapool); - kfifo_free(session->cmds[i]->r2tqueue); - iscsi_pool_free(&session->cmds[i]->r2tpool, - (void**)session->cmds[i]->r2ts); + struct iscsi_cmd_task *ctask = session->cmds[i]; + struct iscsi_tcp_cmd_task *tcp_ctask = ctask->dd_data; + + mempool_destroy(tcp_ctask->datapool); + kfifo_free(tcp_ctask->r2tqueue); + iscsi_pool_free(&tcp_ctask->r2tpool, + (void**)tcp_ctask->r2ts); } return -ENOMEM; } @@ -3228,127 +2157,14 @@ iscsi_r2tpool_free(struct iscsi_session int i; for (i = 0; i < session->cmds_max; i++) { - mempool_destroy(session->cmds[i]->datapool); - kfifo_free(session->cmds[i]->r2tqueue); - iscsi_pool_free(&session->cmds[i]->r2tpool, - (void**)session->cmds[i]->r2ts); - } -} - -static struct scsi_host_template iscsi_sht = { - .name = "iSCSI Initiator over TCP/IP, v." - ISCSI_VERSION_STR, - .queuecommand = iscsi_queuecommand, - .change_queue_depth = iscsi_change_queue_depth, - .can_queue = ISCSI_XMIT_CMDS_MAX - 1, - .sg_tablesize = ISCSI_SG_TABLESIZE, - .cmd_per_lun = ISCSI_DEF_CMD_PER_LUN, - .eh_abort_handler = iscsi_eh_abort, - .eh_host_reset_handler = iscsi_eh_host_reset, - .use_clustering = DISABLE_CLUSTERING, - .proc_name = "iscsi_tcp", - .this_id = -1, -}; - -static struct iscsi_transport iscsi_tcp_transport; + struct iscsi_cmd_task *ctask = session->cmds[i]; + struct iscsi_tcp_cmd_task *tcp_ctask = ctask->dd_data; -static struct iscsi_cls_session * -iscsi_session_create(struct scsi_transport_template *scsit, - uint32_t initial_cmdsn, uint32_t *sid) -{ - struct Scsi_Host *shost; - struct iscsi_session *session; - int cmd_i; - - shost = iscsi_transport_create_session(scsit, &iscsi_tcp_transport); - if (!shost) - return NULL; - - session = iscsi_hostdata(shost->hostdata); - memset(session, 0, sizeof(struct iscsi_session)); - session->host = shost; - session->state = ISCSI_STATE_FREE; - session->mgmtpool_max = ISCSI_MGMT_CMDS_MAX; - session->cmds_max = ISCSI_XMIT_CMDS_MAX; - session->cmdsn = initial_cmdsn; - session->exp_cmdsn = initial_cmdsn + 1; - session->max_cmdsn = initial_cmdsn + 1; - session->max_r2t = 1; - *sid = shost->host_no; - - /* initialize SCSI PDU commands pool */ - if (iscsi_pool_init(&session->cmdpool, session->cmds_max, - (void***)&session->cmds, sizeof(struct iscsi_cmd_task))) - goto cmdpool_alloc_fail; - - /* pre-format cmds pool with ITT */ - for (cmd_i = 0; cmd_i < session->cmds_max; cmd_i++) - session->cmds[cmd_i]->itt = cmd_i; - - spin_lock_init(&session->lock); - INIT_LIST_HEAD(&session->connections); - - /* initialize immediate command pool */ - if (iscsi_pool_init(&session->mgmtpool, session->mgmtpool_max, - (void***)&session->mgmt_cmds, sizeof(struct iscsi_mgmt_task))) - goto mgmtpool_alloc_fail; - - - /* pre-format immediate cmds pool with ITT */ - for (cmd_i = 0; cmd_i < session->mgmtpool_max; cmd_i++) { - session->mgmt_cmds[cmd_i]->itt = ISCSI_MGMT_ITT_OFFSET + cmd_i; - session->mgmt_cmds[cmd_i]->data = kmalloc( - DEFAULT_MAX_RECV_DATA_SEGMENT_LENGTH, GFP_KERNEL); - if (!session->mgmt_cmds[cmd_i]->data) { - int j; - - for (j = 0; j < cmd_i; j++) - kfree(session->mgmt_cmds[j]->data); - goto immdata_alloc_fail; - } - } - - if (iscsi_r2tpool_alloc(session)) - goto r2tpool_alloc_fail; - - return hostdata_session(shost->hostdata); - -r2tpool_alloc_fail: - for (cmd_i = 0; cmd_i < session->mgmtpool_max; cmd_i++) - kfree(session->mgmt_cmds[cmd_i]->data); -immdata_alloc_fail: - iscsi_pool_free(&session->mgmtpool, (void**)session->mgmt_cmds); -mgmtpool_alloc_fail: - iscsi_pool_free(&session->cmdpool, (void**)session->cmds); -cmdpool_alloc_fail: - iscsi_transport_destroy_session(shost); - return NULL; -} - -static void -iscsi_session_destroy(struct iscsi_cls_session *cls_session) -{ - struct Scsi_Host *shost = iscsi_session_to_shost(cls_session); - struct iscsi_session *session = iscsi_hostdata(shost->hostdata); - int cmd_i; - struct iscsi_data_task *dtask, *n; - - for (cmd_i = 0; cmd_i < session->cmds_max; cmd_i++) { - struct iscsi_cmd_task *ctask = session->cmds[cmd_i]; - list_for_each_entry_safe(dtask, n, &ctask->dataqueue, item) { - list_del(&dtask->item); - mempool_free(dtask, ctask->datapool); - } + mempool_destroy(tcp_ctask->datapool); + kfifo_free(tcp_ctask->r2tqueue); + iscsi_pool_free(&tcp_ctask->r2tpool, + (void**)tcp_ctask->r2ts); } - - for (cmd_i = 0; cmd_i < session->mgmtpool_max; cmd_i++) - kfree(session->mgmt_cmds[cmd_i]->data); - - iscsi_r2tpool_free(session); - iscsi_pool_free(&session->mgmtpool, (void**)session->mgmt_cmds); - iscsi_pool_free(&session->cmdpool, (void**)session->cmds); - - iscsi_transport_destroy_session(shost); } static int @@ -3357,23 +2173,14 @@ iscsi_conn_set_param(struct iscsi_cls_co { struct iscsi_conn *conn = cls_conn->dd_data; struct iscsi_session *session = conn->session; - - spin_lock_bh(&session->lock); - if (conn->c_stage != ISCSI_CONN_INITIAL_STAGE && - conn->stop_stage != STOP_CONN_RECOVER) { - printk(KERN_ERR "iscsi_tcp: can not change parameter [%d]\n", - param); - spin_unlock_bh(&session->lock); - return 0; - } - spin_unlock_bh(&session->lock); + struct iscsi_tcp_conn *tcp_conn = conn->dd_data; switch(param) { case ISCSI_PARAM_MAX_RECV_DLENGTH: { - char *saveptr = conn->data; + char *saveptr = tcp_conn->data; gfp_t flags = GFP_KERNEL; - if (conn->data_size >= value) { + if (tcp_conn->data_size >= value) { conn->max_recv_dlength = value; break; } @@ -3384,21 +2191,21 @@ iscsi_conn_set_param(struct iscsi_cls_co spin_unlock_bh(&session->lock); if (value <= PAGE_SIZE) - conn->data = kmalloc(value, flags); + tcp_conn->data = kmalloc(value, flags); else - conn->data = (void*)__get_free_pages(flags, + tcp_conn->data = (void*)__get_free_pages(flags, get_order(value)); - if (conn->data == NULL) { - conn->data = saveptr; + if (tcp_conn->data == NULL) { + tcp_conn->data = saveptr; return -ENOMEM; } - if (conn->data_size <= PAGE_SIZE) + if (tcp_conn->data_size <= PAGE_SIZE) kfree(saveptr); else free_pages((unsigned long)saveptr, - get_order(conn->data_size)); + get_order(tcp_conn->data_size)); conn->max_recv_dlength = value; - conn->data_size = value; + tcp_conn->data_size = value; } break; case ISCSI_PARAM_MAX_XMIT_DLENGTH: @@ -3406,49 +2213,51 @@ iscsi_conn_set_param(struct iscsi_cls_co break; case ISCSI_PARAM_HDRDGST_EN: conn->hdrdgst_en = value; - conn->hdr_size = sizeof(struct iscsi_hdr); + tcp_conn->hdr_size = sizeof(struct iscsi_hdr); if (conn->hdrdgst_en) { - conn->hdr_size += sizeof(__u32); - if (!conn->tx_tfm) - conn->tx_tfm = crypto_alloc_tfm("crc32c", 0); - if (!conn->tx_tfm) + tcp_conn->hdr_size += sizeof(__u32); + if (!tcp_conn->tx_tfm) + tcp_conn->tx_tfm = crypto_alloc_tfm("crc32c", + 0); + if (!tcp_conn->tx_tfm) return -ENOMEM; - if (!conn->rx_tfm) - conn->rx_tfm = crypto_alloc_tfm("crc32c", 0); - if (!conn->rx_tfm) { - crypto_free_tfm(conn->tx_tfm); + if (!tcp_conn->rx_tfm) + tcp_conn->rx_tfm = crypto_alloc_tfm("crc32c", + 0); + if (!tcp_conn->rx_tfm) { + crypto_free_tfm(tcp_conn->tx_tfm); return -ENOMEM; } } else { - if (conn->tx_tfm) - crypto_free_tfm(conn->tx_tfm); - if (conn->rx_tfm) - crypto_free_tfm(conn->rx_tfm); + if (tcp_conn->tx_tfm) + crypto_free_tfm(tcp_conn->tx_tfm); + if (tcp_conn->rx_tfm) + crypto_free_tfm(tcp_conn->rx_tfm); } break; case ISCSI_PARAM_DATADGST_EN: conn->datadgst_en = value; if (conn->datadgst_en) { - if (!conn->data_tx_tfm) - conn->data_tx_tfm = + if (!tcp_conn->data_tx_tfm) + tcp_conn->data_tx_tfm = crypto_alloc_tfm("crc32c", 0); - if (!conn->data_tx_tfm) + if (!tcp_conn->data_tx_tfm) return -ENOMEM; - if (!conn->data_rx_tfm) - conn->data_rx_tfm = + if (!tcp_conn->data_rx_tfm) + tcp_conn->data_rx_tfm = crypto_alloc_tfm("crc32c", 0); - if (!conn->data_rx_tfm) { - crypto_free_tfm(conn->data_tx_tfm); + if (!tcp_conn->data_rx_tfm) { + crypto_free_tfm(tcp_conn->data_tx_tfm); return -ENOMEM; } } else { - if (conn->data_tx_tfm) - crypto_free_tfm(conn->data_tx_tfm); - if (conn->data_rx_tfm) - crypto_free_tfm(conn->data_rx_tfm); + if (tcp_conn->data_tx_tfm) + crypto_free_tfm(tcp_conn->data_tx_tfm); + if (tcp_conn->data_rx_tfm) + crypto_free_tfm(tcp_conn->data_rx_tfm); } - conn->sendpage = conn->datadgst_en ? - sock_no_sendpage : conn->sock->ops->sendpage; + tcp_conn->sendpage = conn->datadgst_en ? + sock_no_sendpage : tcp_conn->sock->ops->sendpage; break; case ISCSI_PARAM_INITIAL_R2T_EN: session->initial_r2t_en = value; @@ -3535,7 +2344,7 @@ iscsi_session_get_param(struct iscsi_cls *value = session->ofmarker_en; break; default: - return ISCSI_ERR_PARAM_NOT_FOUND; + return -EINVAL; } return 0; @@ -3546,6 +2355,8 @@ iscsi_conn_get_param(struct iscsi_cls_co enum iscsi_param param, uint32_t *value) { struct iscsi_conn *conn = cls_conn->dd_data; + struct iscsi_tcp_conn *tcp_conn = conn->dd_data; + struct inet_sock *inet; switch(param) { case ISCSI_PARAM_MAX_RECV_DLENGTH: @@ -3560,17 +2371,67 @@ iscsi_conn_get_param(struct iscsi_cls_co case ISCSI_PARAM_DATADGST_EN: *value = conn->datadgst_en; break; + case ISCSI_PARAM_CONN_PORT: + mutex_lock(&conn->xmitmutex); + if (!tcp_conn->sock) { + mutex_unlock(&conn->xmitmutex); + return -EINVAL; + } + + inet = inet_sk(tcp_conn->sock->sk); + *value = be16_to_cpu(inet->dport); + mutex_unlock(&conn->xmitmutex); default: - return ISCSI_ERR_PARAM_NOT_FOUND; + return -EINVAL; } return 0; } +static int +iscsi_conn_get_str_param(struct iscsi_cls_conn *cls_conn, + enum iscsi_param param, char *buf) +{ + struct iscsi_conn *conn = cls_conn->dd_data; + struct iscsi_tcp_conn *tcp_conn = conn->dd_data; + struct sock *sk; + struct inet_sock *inet; + struct ipv6_pinfo *np; + int len = 0; + + switch (param) { + case ISCSI_PARAM_CONN_ADDRESS: + mutex_lock(&conn->xmitmutex); + if (!tcp_conn->sock) { + mutex_unlock(&conn->xmitmutex); + return -EINVAL; + } + + sk = tcp_conn->sock->sk; + if (sk->sk_family == PF_INET) { + inet = inet_sk(sk); + len = sprintf(buf, "%u.%u.%u.%u\n", + NIPQUAD(inet->daddr)); + } else { + np = inet6_sk(sk); + len = sprintf(buf, + "%04x:%04x:%04x:%04x:%04x:%04x:%04x:%04x\n", + NIP6(np->daddr)); + } + mutex_unlock(&conn->xmitmutex); + break; + default: + return -EINVAL; + } + + return len; +} + static void iscsi_conn_get_stats(struct iscsi_cls_conn *cls_conn, struct iscsi_stats *stats) { struct iscsi_conn *conn = cls_conn->dd_data; + struct iscsi_tcp_conn *tcp_conn = conn->dd_data; stats->txdata_octets = conn->txdata_octets; stats->rxdata_octets = conn->rxdata_octets; @@ -3583,49 +2444,141 @@ iscsi_conn_get_stats(struct iscsi_cls_co stats->tmfrsp_pdus = conn->tmfrsp_pdus_cnt; stats->custom_length = 3; strcpy(stats->custom[0].desc, "tx_sendpage_failures"); - stats->custom[0].value = conn->sendpage_failures_cnt; + stats->custom[0].value = tcp_conn->sendpage_failures_cnt; strcpy(stats->custom[1].desc, "rx_discontiguous_hdr"); - stats->custom[1].value = conn->discontiguous_hdr_cnt; + stats->custom[1].value = tcp_conn->discontiguous_hdr_cnt; strcpy(stats->custom[2].desc, "eh_abort_cnt"); stats->custom[2].value = conn->eh_abort_cnt; } -static int -iscsi_conn_send_pdu(struct iscsi_cls_conn *cls_conn, struct iscsi_hdr *hdr, - char *data, uint32_t data_size) +static struct iscsi_cls_session * +iscsi_tcp_session_create(struct iscsi_transport *iscsit, + struct scsi_transport_template *scsit, + uint32_t initial_cmdsn, uint32_t *hostno) { - struct iscsi_conn *conn = cls_conn->dd_data; - int rc; + struct iscsi_cls_session *cls_session; + struct iscsi_session *session; + uint32_t hn; + int cmd_i; - mutex_lock(&conn->xmitmutex); - rc = iscsi_conn_send_generic(conn, hdr, data, data_size); - mutex_unlock(&conn->xmitmutex); + cls_session = iscsi_session_setup(iscsit, scsit, + sizeof(struct iscsi_tcp_cmd_task), + sizeof(struct iscsi_tcp_mgmt_task), + initial_cmdsn, &hn); + if (!cls_session) + return NULL; + *hostno = hn; - return rc; + session = class_to_transport_session(cls_session); + for (cmd_i = 0; cmd_i < session->cmds_max; cmd_i++) { + struct iscsi_cmd_task *ctask = session->cmds[cmd_i]; + struct iscsi_tcp_cmd_task *tcp_ctask = ctask->dd_data; + + ctask->hdr = &tcp_ctask->hdr; + } + + for (cmd_i = 0; cmd_i < session->mgmtpool_max; cmd_i++) { + struct iscsi_mgmt_task *mtask = session->mgmt_cmds[cmd_i]; + struct iscsi_tcp_mgmt_task *tcp_mtask = mtask->dd_data; + + mtask->hdr = &tcp_mtask->hdr; + } + + if (iscsi_r2tpool_alloc(class_to_transport_session(cls_session))) + goto r2tpool_alloc_fail; + + return cls_session; + +r2tpool_alloc_fail: + iscsi_session_teardown(cls_session); + return NULL; } +static void iscsi_tcp_session_destroy(struct iscsi_cls_session *cls_session) +{ + struct iscsi_session *session = class_to_transport_session(cls_session); + struct iscsi_data_task *dtask, *n; + int cmd_i; + + for (cmd_i = 0; cmd_i < session->cmds_max; cmd_i++) { + struct iscsi_cmd_task *ctask = session->cmds[cmd_i]; + struct iscsi_tcp_cmd_task *tcp_ctask = ctask->dd_data; + + list_for_each_entry_safe(dtask, n, &tcp_ctask->dataqueue, + item) { + list_del(&dtask->item); + mempool_free(dtask, tcp_ctask->datapool); + } + } + + iscsi_r2tpool_free(class_to_transport_session(cls_session)); + iscsi_session_teardown(cls_session); +} + +static struct scsi_host_template iscsi_sht = { + .name = "iSCSI Initiator over TCP/IP, v." + ISCSI_VERSION_STR, + .queuecommand = iscsi_queuecommand, + .change_queue_depth = iscsi_change_queue_depth, + .can_queue = ISCSI_XMIT_CMDS_MAX - 1, + .sg_tablesize = ISCSI_SG_TABLESIZE, + .cmd_per_lun = ISCSI_DEF_CMD_PER_LUN, + .eh_abort_handler = iscsi_eh_abort, + .eh_host_reset_handler = iscsi_eh_host_reset, + .use_clustering = DISABLE_CLUSTERING, + .proc_name = "iscsi_tcp", + .this_id = -1, +}; + static struct iscsi_transport iscsi_tcp_transport = { .owner = THIS_MODULE, .name = "tcp", .caps = CAP_RECOVERY_L0 | CAP_MULTI_R2T | CAP_HDRDGST | CAP_DATADGST, + .param_mask = ISCSI_MAX_RECV_DLENGTH | + ISCSI_MAX_XMIT_DLENGTH | + ISCSI_HDRDGST_EN | + ISCSI_DATADGST_EN | + ISCSI_INITIAL_R2T_EN | + ISCSI_MAX_R2T | + ISCSI_IMM_DATA_EN | + ISCSI_FIRST_BURST | + ISCSI_MAX_BURST | + ISCSI_PDU_INORDER_EN | + ISCSI_DATASEQ_INORDER_EN | + ISCSI_ERL | + ISCSI_CONN_PORT | + ISCSI_CONN_ADDRESS, .host_template = &iscsi_sht, - .hostdata_size = sizeof(struct iscsi_session), .conndata_size = sizeof(struct iscsi_conn), .max_conn = 1, .max_cmd_len = ISCSI_TCP_MAX_CMD_LEN, - .create_session = iscsi_session_create, - .destroy_session = iscsi_session_destroy, - .create_conn = iscsi_conn_create, - .bind_conn = iscsi_conn_bind, - .destroy_conn = iscsi_conn_destroy, + /* session management */ + .create_session = iscsi_tcp_session_create, + .destroy_session = iscsi_tcp_session_destroy, + /* connection management */ + .create_conn = iscsi_tcp_conn_create, + .bind_conn = iscsi_tcp_conn_bind, + .destroy_conn = iscsi_tcp_conn_destroy, .set_param = iscsi_conn_set_param, .get_conn_param = iscsi_conn_get_param, + .get_conn_str_param = iscsi_conn_get_str_param, .get_session_param = iscsi_session_get_param, .start_conn = iscsi_conn_start, .stop_conn = iscsi_conn_stop, + /* these are called as part of conn recovery */ + .suspend_conn_recv = iscsi_tcp_suspend_conn_rx, + .terminate_conn = iscsi_tcp_terminate_conn, + /* IO */ .send_pdu = iscsi_conn_send_pdu, .get_stats = iscsi_conn_get_stats, + .init_cmd_task = iscsi_tcp_cmd_init, + .init_mgmt_task = iscsi_tcp_mgmt_init, + .xmit_cmd_task = iscsi_tcp_ctask_xmit, + .xmit_mgmt_task = iscsi_tcp_mtask_xmit, + .cleanup_cmd_task = iscsi_tcp_cleanup_ctask, + /* recovery */ + .session_recovery_timedout = iscsi_session_recovery_timedout, }; static int __init diff -puN drivers/scsi/iscsi_tcp.h~git-scsi-misc drivers/scsi/iscsi_tcp.h --- devel/drivers/scsi/iscsi_tcp.h~git-scsi-misc 2006-04-30 00:30:24.000000000 -0700 +++ devel-akpm/drivers/scsi/iscsi_tcp.h 2006-04-30 00:30:24.000000000 -0700 @@ -2,7 +2,8 @@ * iSCSI Initiator TCP Transport * Copyright (C) 2004 Dmitry Yusupov * Copyright (C) 2004 Alex Aizman - * Copyright (C) 2005 Mike Christie + * Copyright (C) 2005 - 2006 Mike Christie + * Copyright (C) 2006 Red Hat, Inc. All rights reserved. * maintained by open-iscsi@googlegroups.com * * This program is free software; you can redistribute it and/or modify @@ -21,20 +22,7 @@ #ifndef ISCSI_TCP_H #define ISCSI_TCP_H -/* Session's states */ -#define ISCSI_STATE_FREE 1 -#define ISCSI_STATE_LOGGED_IN 2 -#define ISCSI_STATE_FAILED 3 -#define ISCSI_STATE_TERMINATE 4 - -/* Connection's states */ -#define ISCSI_CONN_INITIAL_STAGE 0 -#define ISCSI_CONN_STARTED 1 -#define ISCSI_CONN_STOPPED 2 -#define ISCSI_CONN_CLEANUP_WAIT 3 - -/* Connection suspend "bit" */ -#define SUSPEND_BIT 1 +#include /* Socket's Receive state machine */ #define IN_PROGRESS_WAIT_HEADER 0x0 @@ -42,12 +30,6 @@ #define IN_PROGRESS_DATA_RECV 0x2 #define IN_PROGRESS_DDIGEST_RECV 0x3 -/* Task Mgmt states */ -#define TMABORT_INITIAL 0x0 -#define TMABORT_SUCCESS 0x1 -#define TMABORT_FAILED 0x2 -#define TMABORT_TIMEDOUT 0x3 - /* xmit state machine */ #define XMSTATE_IDLE 0x0 #define XMSTATE_R_HDR 0x1 @@ -62,34 +44,14 @@ #define XMSTATE_W_PAD 0x200 #define XMSTATE_DATA_DIGEST 0x400 -#define ISCSI_CONN_MAX 1 #define ISCSI_CONN_RCVBUF_MIN 262144 #define ISCSI_CONN_SNDBUF_MIN 262144 #define ISCSI_PAD_LEN 4 #define ISCSI_R2T_MAX 16 -#define ISCSI_XMIT_CMDS_MAX 128 /* must be power of 2 */ -#define ISCSI_MGMT_CMDS_MAX 32 /* must be power of 2 */ -#define ISCSI_MGMT_ITT_OFFSET 0xa00 #define ISCSI_SG_TABLESIZE SG_ALL -#define ISCSI_DEF_CMD_PER_LUN 32 -#define ISCSI_MAX_CMD_PER_LUN 128 #define ISCSI_TCP_MAX_CMD_LEN 16 -#define ITT_MASK (0xfff) -#define CID_SHIFT 12 -#define CID_MASK (0xffff< +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +struct iscsi_session * +class_to_transport_session(struct iscsi_cls_session *cls_session) +{ + struct Scsi_Host *shost = iscsi_session_to_shost(cls_session); + return iscsi_hostdata(shost->hostdata); +} +EXPORT_SYMBOL_GPL(class_to_transport_session); + +#define INVALID_SN_DELTA 0xffff + +int +iscsi_check_assign_cmdsn(struct iscsi_session *session, struct iscsi_nopin *hdr) +{ + uint32_t max_cmdsn = be32_to_cpu(hdr->max_cmdsn); + uint32_t exp_cmdsn = be32_to_cpu(hdr->exp_cmdsn); + + if (max_cmdsn < exp_cmdsn -1 && + max_cmdsn > exp_cmdsn - INVALID_SN_DELTA) + return ISCSI_ERR_MAX_CMDSN; + if (max_cmdsn > session->max_cmdsn || + max_cmdsn < session->max_cmdsn - INVALID_SN_DELTA) + session->max_cmdsn = max_cmdsn; + if (exp_cmdsn > session->exp_cmdsn || + exp_cmdsn < session->exp_cmdsn - INVALID_SN_DELTA) + session->exp_cmdsn = exp_cmdsn; + + return 0; +} +EXPORT_SYMBOL_GPL(iscsi_check_assign_cmdsn); + +void iscsi_prep_unsolicit_data_pdu(struct iscsi_cmd_task *ctask, + struct iscsi_data *hdr, + int transport_data_cnt) +{ + struct iscsi_conn *conn = ctask->conn; + + memset(hdr, 0, sizeof(struct iscsi_data)); + hdr->ttt = cpu_to_be32(ISCSI_RESERVED_TAG); + hdr->datasn = cpu_to_be32(ctask->unsol_datasn); + ctask->unsol_datasn++; + hdr->opcode = ISCSI_OP_SCSI_DATA_OUT; + memcpy(hdr->lun, ctask->hdr->lun, sizeof(hdr->lun)); + + hdr->itt = ctask->hdr->itt; + hdr->exp_statsn = cpu_to_be32(conn->exp_statsn); + + hdr->offset = cpu_to_be32(ctask->total_length - + transport_data_cnt - + ctask->unsol_count); + + if (ctask->unsol_count > conn->max_xmit_dlength) { + hton24(hdr->dlength, conn->max_xmit_dlength); + ctask->data_count = conn->max_xmit_dlength; + hdr->flags = 0; + } else { + hton24(hdr->dlength, ctask->unsol_count); + ctask->data_count = ctask->unsol_count; + hdr->flags = ISCSI_FLAG_CMD_FINAL; + } +} +EXPORT_SYMBOL_GPL(iscsi_prep_unsolicit_data_pdu); + +/** + * iscsi_prep_scsi_cmd_pdu - prep iscsi scsi cmd pdu + * @ctask: iscsi cmd task + * + * Prep basic iSCSI PDU fields for a scsi cmd pdu. The LLD should set + * fields like dlength or final based on how much data it sends + */ +static void iscsi_prep_scsi_cmd_pdu(struct iscsi_cmd_task *ctask) +{ + struct iscsi_conn *conn = ctask->conn; + struct iscsi_session *session = conn->session; + struct iscsi_cmd *hdr = ctask->hdr; + struct scsi_cmnd *sc = ctask->sc; + + hdr->opcode = ISCSI_OP_SCSI_CMD; + hdr->flags = ISCSI_ATTR_SIMPLE; + int_to_scsilun(sc->device->lun, (struct scsi_lun *)hdr->lun); + hdr->itt = ctask->itt | (conn->id << ISCSI_CID_SHIFT) | + (session->age << ISCSI_AGE_SHIFT); + hdr->data_length = cpu_to_be32(sc->request_bufflen); + hdr->cmdsn = cpu_to_be32(session->cmdsn); + session->cmdsn++; + hdr->exp_statsn = cpu_to_be32(conn->exp_statsn); + memcpy(hdr->cdb, sc->cmnd, sc->cmd_len); + memset(&hdr->cdb[sc->cmd_len], 0, MAX_COMMAND_SIZE - sc->cmd_len); + + if (sc->sc_data_direction == DMA_TO_DEVICE) { + hdr->flags |= ISCSI_FLAG_CMD_WRITE; + /* + * Write counters: + * + * imm_count bytes to be sent right after + * SCSI PDU Header + * + * unsol_count bytes(as Data-Out) to be sent + * without R2T ack right after + * immediate data + * + * r2t_data_count bytes to be sent via R2T ack's + * + * pad_count bytes to be sent as zero-padding + */ + ctask->imm_count = 0; + ctask->unsol_count = 0; + ctask->unsol_datasn = 0; + + if (session->imm_data_en) { + if (ctask->total_length >= session->first_burst) + ctask->imm_count = min(session->first_burst, + conn->max_xmit_dlength); + else + ctask->imm_count = min(ctask->total_length, + conn->max_xmit_dlength); + hton24(ctask->hdr->dlength, ctask->imm_count); + } else + zero_data(ctask->hdr->dlength); + + if (!session->initial_r2t_en) + ctask->unsol_count = min(session->first_burst, + ctask->total_length) - ctask->imm_count; + if (!ctask->unsol_count) + /* No unsolicit Data-Out's */ + ctask->hdr->flags |= ISCSI_FLAG_CMD_FINAL; + } else { + ctask->datasn = 0; + hdr->flags |= ISCSI_FLAG_CMD_FINAL; + zero_data(hdr->dlength); + + if (sc->sc_data_direction == DMA_FROM_DEVICE) + hdr->flags |= ISCSI_FLAG_CMD_READ; + } + + conn->scsicmd_pdus_cnt++; +} +EXPORT_SYMBOL_GPL(iscsi_prep_scsi_cmd_pdu); + +/** + * iscsi_complete_command - return command back to scsi-ml + * @session: iscsi session + * @ctask: iscsi cmd task + * + * Must be called with session lock. + * This function returns the scsi command to scsi-ml and returns + * the cmd task to the pool of available cmd tasks. + */ +static void iscsi_complete_command(struct iscsi_session *session, + struct iscsi_cmd_task *ctask) +{ + struct scsi_cmnd *sc = ctask->sc; + + ctask->sc = NULL; + list_del_init(&ctask->running); + __kfifo_put(session->cmdpool.queue, (void*)&ctask, sizeof(void*)); + sc->scsi_done(sc); +} + +/** + * iscsi_cmd_rsp - SCSI Command Response processing + * @conn: iscsi connection + * @hdr: iscsi header + * @ctask: scsi command task + * @data: cmd data buffer + * @datalen: len of buffer + * + * iscsi_cmd_rsp sets up the scsi_cmnd fields based on the PDU and + * then completes the command and task. + **/ +static int iscsi_scsi_cmd_rsp(struct iscsi_conn *conn, struct iscsi_hdr *hdr, + struct iscsi_cmd_task *ctask, char *data, + int datalen) +{ + int rc; + struct iscsi_cmd_rsp *rhdr = (struct iscsi_cmd_rsp *)hdr; + struct iscsi_session *session = conn->session; + struct scsi_cmnd *sc = ctask->sc; + + rc = iscsi_check_assign_cmdsn(session, (struct iscsi_nopin*)rhdr); + if (rc) { + sc->result = DID_ERROR << 16; + goto out; + } + + conn->exp_statsn = be32_to_cpu(rhdr->statsn) + 1; + + sc->result = (DID_OK << 16) | rhdr->cmd_status; + + if (rhdr->response != ISCSI_STATUS_CMD_COMPLETED) { + sc->result = DID_ERROR << 16; + goto out; + } + + if (rhdr->cmd_status == SAM_STAT_CHECK_CONDITION) { + int senselen; + + if (datalen < 2) { +invalid_datalen: + printk(KERN_ERR "Got CHECK_CONDITION but invalid " + "data buffer size of %d\n", datalen); + sc->result = DID_BAD_TARGET << 16; + goto out; + } + + senselen = (data[0] << 8) | data[1]; + if (datalen < senselen) + goto invalid_datalen; + + memcpy(sc->sense_buffer, data + 2, + min(senselen, SCSI_SENSE_BUFFERSIZE)); + debug_scsi("copied %d bytes of sense\n", + min(senselen, SCSI_SENSE_BUFFERSIZE)); + } + + if (sc->sc_data_direction == DMA_TO_DEVICE) + goto out; + + if (rhdr->flags & ISCSI_FLAG_CMD_UNDERFLOW) { + int res_count = be32_to_cpu(rhdr->residual_count); + + if (res_count > 0 && res_count <= sc->request_bufflen) + sc->resid = res_count; + else + sc->result = (DID_BAD_TARGET << 16) | rhdr->cmd_status; + } else if (rhdr->flags & ISCSI_FLAG_CMD_BIDI_UNDERFLOW) + sc->result = (DID_BAD_TARGET << 16) | rhdr->cmd_status; + else if (rhdr->flags & ISCSI_FLAG_CMD_OVERFLOW) + sc->resid = be32_to_cpu(rhdr->residual_count); + +out: + debug_scsi("done [sc %lx res %d itt 0x%x]\n", + (long)sc, sc->result, ctask->itt); + conn->scsirsp_pdus_cnt++; + + iscsi_complete_command(conn->session, ctask); + return rc; +} + +/** + * __iscsi_complete_pdu - complete pdu + * @conn: iscsi conn + * @hdr: iscsi header + * @data: data buffer + * @datalen: len of data buffer + * + * Completes pdu processing by freeing any resources allocated at + * queuecommand or send generic. session lock must be held and verify + * itt must have been called. + */ +int __iscsi_complete_pdu(struct iscsi_conn *conn, struct iscsi_hdr *hdr, + char *data, int datalen) +{ + struct iscsi_session *session = conn->session; + int opcode = hdr->opcode & ISCSI_OPCODE_MASK, rc = 0; + struct iscsi_cmd_task *ctask; + struct iscsi_mgmt_task *mtask; + uint32_t itt; + + if (hdr->itt != cpu_to_be32(ISCSI_RESERVED_TAG)) + itt = hdr->itt & ISCSI_ITT_MASK; + else + itt = hdr->itt; + + if (itt < session->cmds_max) { + ctask = session->cmds[itt]; + + debug_scsi("cmdrsp [op 0x%x cid %d itt 0x%x len %d]\n", + opcode, conn->id, ctask->itt, datalen); + + switch(opcode) { + case ISCSI_OP_SCSI_CMD_RSP: + BUG_ON((void*)ctask != ctask->sc->SCp.ptr); + rc = iscsi_scsi_cmd_rsp(conn, hdr, ctask, data, + datalen); + break; + case ISCSI_OP_SCSI_DATA_IN: + BUG_ON((void*)ctask != ctask->sc->SCp.ptr); + if (hdr->flags & ISCSI_FLAG_DATA_STATUS) { + conn->scsirsp_pdus_cnt++; + iscsi_complete_command(session, ctask); + } + break; + case ISCSI_OP_R2T: + /* LLD handles this for now */ + break; + default: + rc = ISCSI_ERR_BAD_OPCODE; + break; + } + } else if (itt >= ISCSI_MGMT_ITT_OFFSET && + itt < ISCSI_MGMT_ITT_OFFSET + session->mgmtpool_max) { + mtask = session->mgmt_cmds[itt - ISCSI_MGMT_ITT_OFFSET]; + + debug_scsi("immrsp [op 0x%x cid %d itt 0x%x len %d]\n", + opcode, conn->id, mtask->itt, datalen); + + switch(opcode) { + case ISCSI_OP_LOGIN_RSP: + case ISCSI_OP_TEXT_RSP: + case ISCSI_OP_LOGOUT_RSP: + rc = iscsi_check_assign_cmdsn(session, + (struct iscsi_nopin*)hdr); + if (rc) + break; + + rc = iscsi_recv_pdu(conn->cls_conn, hdr, data, datalen); + list_del(&mtask->running); + if (conn->login_mtask != mtask) + __kfifo_put(session->mgmtpool.queue, + (void*)&mtask, sizeof(void*)); + break; + case ISCSI_OP_SCSI_TMFUNC_RSP: + rc = iscsi_check_assign_cmdsn(session, + (struct iscsi_nopin*)hdr); + if (rc) + break; + + if (datalen) { + rc = ISCSI_ERR_PROTO; + break; + } + conn->tmfrsp_pdus_cnt++; + if (conn->tmabort_state == TMABORT_INITIAL) { + conn->tmabort_state = + ((struct iscsi_tm_rsp *)hdr)-> + response == ISCSI_TMF_RSP_COMPLETE ? + TMABORT_SUCCESS:TMABORT_FAILED; + /* unblock eh_abort() */ + wake_up(&conn->ehwait); + } + break; + case ISCSI_OP_NOOP_IN: + if (hdr->ttt != ISCSI_RESERVED_TAG) { + rc = ISCSI_ERR_PROTO; + break; + } + rc = iscsi_check_assign_cmdsn(session, + (struct iscsi_nopin*)hdr); + if (rc) + break; + conn->exp_statsn = be32_to_cpu(hdr->statsn) + 1; + + rc = iscsi_recv_pdu(conn->cls_conn, hdr, data, datalen); + list_del(&mtask->running); + if (conn->login_mtask != mtask) + __kfifo_put(session->mgmtpool.queue, + (void*)&mtask, sizeof(void*)); + break; + default: + rc = ISCSI_ERR_BAD_OPCODE; + break; + } + } else if (itt == ISCSI_RESERVED_TAG) { + switch(opcode) { + case ISCSI_OP_NOOP_IN: + if (!datalen) { + rc = iscsi_check_assign_cmdsn(session, + (struct iscsi_nopin*)hdr); + if (!rc && hdr->ttt != ISCSI_RESERVED_TAG) + rc = iscsi_recv_pdu(conn->cls_conn, + hdr, NULL, 0); + } else + rc = ISCSI_ERR_PROTO; + break; + case ISCSI_OP_REJECT: + /* we need sth like iscsi_reject_rsp()*/ + case ISCSI_OP_ASYNC_EVENT: + /* we need sth like iscsi_async_event_rsp() */ + rc = ISCSI_ERR_BAD_OPCODE; + break; + default: + rc = ISCSI_ERR_BAD_OPCODE; + break; + } + } else + rc = ISCSI_ERR_BAD_ITT; + + return rc; +} +EXPORT_SYMBOL_GPL(__iscsi_complete_pdu); + +int iscsi_complete_pdu(struct iscsi_conn *conn, struct iscsi_hdr *hdr, + char *data, int datalen) +{ + int rc; + + spin_lock(&conn->session->lock); + rc = __iscsi_complete_pdu(conn, hdr, data, datalen); + spin_unlock(&conn->session->lock); + return rc; +} +EXPORT_SYMBOL_GPL(iscsi_complete_pdu); + +/* verify itt (itt encoding: age+cid+itt) */ +int iscsi_verify_itt(struct iscsi_conn *conn, struct iscsi_hdr *hdr, + uint32_t *ret_itt) +{ + struct iscsi_session *session = conn->session; + struct iscsi_cmd_task *ctask; + uint32_t itt; + + if (hdr->itt != cpu_to_be32(ISCSI_RESERVED_TAG)) { + if ((hdr->itt & ISCSI_AGE_MASK) != + (session->age << ISCSI_AGE_SHIFT)) { + printk(KERN_ERR "iscsi_tcp: received itt %x expected " + "session age (%x)\n", hdr->itt, + session->age & ISCSI_AGE_MASK); + return ISCSI_ERR_BAD_ITT; + } + + if ((hdr->itt & ISCSI_CID_MASK) != + (conn->id << ISCSI_CID_SHIFT)) { + printk(KERN_ERR "iscsi_tcp: received itt %x, expected " + "CID (%x)\n", hdr->itt, conn->id); + return ISCSI_ERR_BAD_ITT; + } + itt = hdr->itt & ISCSI_ITT_MASK; + } else + itt = hdr->itt; + + if (itt < session->cmds_max) { + ctask = session->cmds[itt]; + + if (!ctask->sc) { + printk(KERN_INFO "iscsi_tcp: dropping ctask with " + "itt 0x%x\n", ctask->itt); + /* force drop */ + return ISCSI_ERR_NO_SCSI_CMD; + } + + if (ctask->sc->SCp.phase != session->age) { + printk(KERN_ERR "iscsi_tcp: ctask's session age %d, " + "expected %d\n", ctask->sc->SCp.phase, + session->age); + return ISCSI_ERR_SESSION_FAILED; + } + } + + *ret_itt = itt; + return 0; +} +EXPORT_SYMBOL_GPL(iscsi_verify_itt); + +void iscsi_conn_failure(struct iscsi_conn *conn, enum iscsi_err err) +{ + struct iscsi_session *session = conn->session; + unsigned long flags; + + spin_lock_irqsave(&session->lock, flags); + if (session->conn_cnt == 1 || session->leadconn == conn) + session->state = ISCSI_STATE_FAILED; + spin_unlock_irqrestore(&session->lock, flags); + set_bit(ISCSI_SUSPEND_BIT, &conn->suspend_tx); + set_bit(ISCSI_SUSPEND_BIT, &conn->suspend_rx); + iscsi_conn_error(conn->cls_conn, err); +} +EXPORT_SYMBOL_GPL(iscsi_conn_failure); + +/** + * iscsi_data_xmit - xmit any command into the scheduled connection + * @conn: iscsi connection + * + * Notes: + * The function can return -EAGAIN in which case the caller must + * re-schedule it again later or recover. '0' return code means + * successful xmit. + **/ +static int iscsi_data_xmit(struct iscsi_conn *conn) +{ + struct iscsi_transport *tt; + + if (unlikely(conn->suspend_tx)) { + debug_scsi("conn %d Tx suspended!\n", conn->id); + return 0; + } + tt = conn->session->tt; + + /* + * Transmit in the following order: + * + * 1) un-finished xmit (ctask or mtask) + * 2) immediate control PDUs + * 3) write data + * 4) SCSI commands + * 5) non-immediate control PDUs + * + * No need to lock around __kfifo_get as long as + * there's one producer and one consumer. + */ + + BUG_ON(conn->ctask && conn->mtask); + + if (conn->ctask) { + if (tt->xmit_cmd_task(conn, conn->ctask)) + goto again; + /* done with this in-progress ctask */ + conn->ctask = NULL; + } + if (conn->mtask) { + if (tt->xmit_mgmt_task(conn, conn->mtask)) + goto again; + /* done with this in-progress mtask */ + conn->mtask = NULL; + } + + /* process immediate first */ + if (unlikely(__kfifo_len(conn->immqueue))) { + while (__kfifo_get(conn->immqueue, (void*)&conn->mtask, + sizeof(void*))) { + list_add_tail(&conn->mtask->running, + &conn->mgmt_run_list); + if (tt->xmit_mgmt_task(conn, conn->mtask)) + goto again; + } + /* done with this mtask */ + conn->mtask = NULL; + } + + /* process command queue */ + while (__kfifo_get(conn->xmitqueue, (void*)&conn->ctask, + sizeof(void*))) { + /* + * iscsi tcp may readd the task to the xmitqueue to send + * write data + */ + if (list_empty(&conn->ctask->running)) + list_add_tail(&conn->ctask->running, &conn->run_list); + if (tt->xmit_cmd_task(conn, conn->ctask)) + goto again; + } + /* done with this ctask */ + conn->ctask = NULL; + + /* process the rest control plane PDUs, if any */ + if (unlikely(__kfifo_len(conn->mgmtqueue))) { + while (__kfifo_get(conn->mgmtqueue, (void*)&conn->mtask, + sizeof(void*))) { + list_add_tail(&conn->mtask->running, + &conn->mgmt_run_list); + if (tt->xmit_mgmt_task(conn, conn->mtask)) + goto again; + } + /* done with this mtask */ + conn->mtask = NULL; + } + + return 0; + +again: + if (unlikely(conn->suspend_tx)) + return 0; + + return -EAGAIN; +} + +static void iscsi_xmitworker(void *data) +{ + struct iscsi_conn *conn = data; + + /* + * serialize Xmit worker on a per-connection basis. + */ + mutex_lock(&conn->xmitmutex); + if (iscsi_data_xmit(conn)) + scsi_queue_work(conn->session->host, &conn->xmitwork); + mutex_unlock(&conn->xmitmutex); +} + +enum { + FAILURE_BAD_HOST = 1, + FAILURE_SESSION_FAILED, + FAILURE_SESSION_FREED, + FAILURE_WINDOW_CLOSED, + FAILURE_SESSION_TERMINATE, + FAILURE_SESSION_RECOVERY_TIMEOUT, +}; + +int iscsi_queuecommand(struct scsi_cmnd *sc, void (*done)(struct scsi_cmnd *)) +{ + struct Scsi_Host *host; + int reason = 0; + struct iscsi_session *session; + struct iscsi_conn *conn; + struct iscsi_cmd_task *ctask = NULL; + + sc->scsi_done = done; + sc->result = 0; + + host = sc->device->host; + session = iscsi_hostdata(host->hostdata); + + spin_lock(&session->lock); + + if (session->state != ISCSI_STATE_LOGGED_IN) { + if (session->recovery_failed) { + reason = FAILURE_SESSION_RECOVERY_TIMEOUT; + goto fault; + } else if (session->state == ISCSI_STATE_FAILED) { + reason = FAILURE_SESSION_FAILED; + goto reject; + } else if (session->state == ISCSI_STATE_TERMINATE) { + reason = FAILURE_SESSION_TERMINATE; + goto fault; + } + reason = FAILURE_SESSION_FREED; + goto fault; + } + + /* + * Check for iSCSI window and take care of CmdSN wrap-around + */ + if ((int)(session->max_cmdsn - session->cmdsn) < 0) { + reason = FAILURE_WINDOW_CLOSED; + goto reject; + } + + conn = session->leadconn; + + __kfifo_get(session->cmdpool.queue, (void*)&ctask, sizeof(void*)); + sc->SCp.phase = session->age; + sc->SCp.ptr = (char *)ctask; + + ctask->mtask = NULL; + ctask->conn = conn; + ctask->sc = sc; + INIT_LIST_HEAD(&ctask->running); + ctask->total_length = sc->request_bufflen; + iscsi_prep_scsi_cmd_pdu(ctask); + + session->tt->init_cmd_task(ctask); + + __kfifo_put(conn->xmitqueue, (void*)&ctask, sizeof(void*)); + debug_scsi( + "ctask enq [%s cid %d sc %lx itt 0x%x len %d cmdsn %d win %d]\n", + sc->sc_data_direction == DMA_TO_DEVICE ? "write" : "read", + conn->id, (long)sc, ctask->itt, sc->request_bufflen, + session->cmdsn, session->max_cmdsn - session->exp_cmdsn + 1); + spin_unlock(&session->lock); + + scsi_queue_work(host, &conn->xmitwork); + return 0; + +reject: + spin_unlock(&session->lock); + debug_scsi("cmd 0x%x rejected (%d)\n", sc->cmnd[0], reason); + return SCSI_MLQUEUE_HOST_BUSY; + +fault: + spin_unlock(&session->lock); + printk(KERN_ERR "iscsi_tcp: cmd 0x%x is not queued (%d)\n", + sc->cmnd[0], reason); + sc->result = (DID_NO_CONNECT << 16); + sc->resid = sc->request_bufflen; + sc->scsi_done(sc); + return 0; +} +EXPORT_SYMBOL_GPL(iscsi_queuecommand); + +int iscsi_change_queue_depth(struct scsi_device *sdev, int depth) +{ + if (depth > ISCSI_MAX_CMD_PER_LUN) + depth = ISCSI_MAX_CMD_PER_LUN; + scsi_adjust_queue_depth(sdev, scsi_get_tag_type(sdev), depth); + return sdev->queue_depth; +} +EXPORT_SYMBOL_GPL(iscsi_change_queue_depth); + +static int +iscsi_conn_send_generic(struct iscsi_conn *conn, struct iscsi_hdr *hdr, + char *data, uint32_t data_size) +{ + struct iscsi_session *session = conn->session; + struct iscsi_nopout *nop = (struct iscsi_nopout *)hdr; + struct iscsi_mgmt_task *mtask; + + spin_lock_bh(&session->lock); + if (session->state == ISCSI_STATE_TERMINATE) { + spin_unlock_bh(&session->lock); + return -EPERM; + } + if (hdr->opcode == (ISCSI_OP_LOGIN | ISCSI_OP_IMMEDIATE) || + hdr->opcode == (ISCSI_OP_TEXT | ISCSI_OP_IMMEDIATE)) + /* + * Login and Text are sent serially, in + * request-followed-by-response sequence. + * Same mtask can be used. Same ITT must be used. + * Note that login_mtask is preallocated at conn_create(). + */ + mtask = conn->login_mtask; + else { + BUG_ON(conn->c_stage == ISCSI_CONN_INITIAL_STAGE); + BUG_ON(conn->c_stage == ISCSI_CONN_STOPPED); + + if (!__kfifo_get(session->mgmtpool.queue, + (void*)&mtask, sizeof(void*))) { + spin_unlock_bh(&session->lock); + return -ENOSPC; + } + } + + /* + * pre-format CmdSN and ExpStatSN for outgoing PDU. + */ + if (hdr->itt != cpu_to_be32(ISCSI_RESERVED_TAG)) { + hdr->itt = mtask->itt | (conn->id << ISCSI_CID_SHIFT) | + (session->age << ISCSI_AGE_SHIFT); + nop->cmdsn = cpu_to_be32(session->cmdsn); + if (conn->c_stage == ISCSI_CONN_STARTED && + !(hdr->opcode & ISCSI_OP_IMMEDIATE)) + session->cmdsn++; + } else + /* do not advance CmdSN */ + nop->cmdsn = cpu_to_be32(session->cmdsn); + + nop->exp_statsn = cpu_to_be32(conn->exp_statsn); + + if (data_size) { + memcpy(mtask->data, data, data_size); + mtask->data_count = data_size; + } else + mtask->data_count = 0; + + INIT_LIST_HEAD(&mtask->running); + memcpy(mtask->hdr, hdr, sizeof(struct iscsi_hdr)); + if (session->tt->init_mgmt_task) + session->tt->init_mgmt_task(conn, mtask, data, data_size); + spin_unlock_bh(&session->lock); + + debug_scsi("mgmtpdu [op 0x%x hdr->itt 0x%x datalen %d]\n", + hdr->opcode, hdr->itt, data_size); + + /* + * since send_pdu() could be called at least from two contexts, + * we need to serialize __kfifo_put, so we don't have to take + * additional lock on fast data-path + */ + if (hdr->opcode & ISCSI_OP_IMMEDIATE) + __kfifo_put(conn->immqueue, (void*)&mtask, sizeof(void*)); + else + __kfifo_put(conn->mgmtqueue, (void*)&mtask, sizeof(void*)); + + scsi_queue_work(session->host, &conn->xmitwork); + return 0; +} + +int iscsi_conn_send_pdu(struct iscsi_cls_conn *cls_conn, struct iscsi_hdr *hdr, + char *data, uint32_t data_size) +{ + struct iscsi_conn *conn = cls_conn->dd_data; + int rc; + + mutex_lock(&conn->xmitmutex); + rc = iscsi_conn_send_generic(conn, hdr, data, data_size); + mutex_unlock(&conn->xmitmutex); + + return rc; +} +EXPORT_SYMBOL_GPL(iscsi_conn_send_pdu); + +void iscsi_session_recovery_timedout(struct iscsi_cls_session *cls_session) +{ + struct iscsi_session *session = class_to_transport_session(cls_session); + struct iscsi_conn *conn = session->leadconn; + + spin_lock_bh(&session->lock); + if (session->state != ISCSI_STATE_LOGGED_IN) { + session->recovery_failed = 1; + if (conn) + wake_up(&conn->ehwait); + } + spin_unlock_bh(&session->lock); +} +EXPORT_SYMBOL_GPL(iscsi_session_recovery_timedout); + +int iscsi_eh_host_reset(struct scsi_cmnd *sc) +{ + struct Scsi_Host *host = sc->device->host; + struct iscsi_session *session = iscsi_hostdata(host->hostdata); + struct iscsi_conn *conn = session->leadconn; + int fail_session = 0; + + spin_lock_bh(&session->lock); + if (session->state == ISCSI_STATE_TERMINATE) { +failed: + debug_scsi("failing host reset: session terminated " + "[CID %d age %d]", conn->id, session->age); + spin_unlock_bh(&session->lock); + return FAILED; + } + + if (sc->SCp.phase == session->age) { + debug_scsi("failing connection CID %d due to SCSI host reset", + conn->id); + fail_session = 1; + } + spin_unlock_bh(&session->lock); + + /* + * we drop the lock here but the leadconn cannot be destoyed while + * we are in the scsi eh + */ + if (fail_session) { + iscsi_conn_failure(conn, ISCSI_ERR_CONN_FAILED); + /* + * if userspace cannot respond then we must kick this off + * here for it + */ + iscsi_start_session_recovery(session, conn, STOP_CONN_RECOVER); + } + + debug_scsi("iscsi_eh_host_reset wait for relogin\n"); + wait_event_interruptible(conn->ehwait, + session->state == ISCSI_STATE_TERMINATE || + session->state == ISCSI_STATE_LOGGED_IN || + session->recovery_failed); + if (signal_pending(current)) + flush_signals(current); + + spin_lock_bh(&session->lock); + if (session->state == ISCSI_STATE_LOGGED_IN) + printk(KERN_INFO "host reset succeeded\n"); + else + goto failed; + spin_unlock_bh(&session->lock); + + return SUCCESS; +} +EXPORT_SYMBOL_GPL(iscsi_eh_host_reset); + +static void iscsi_tmabort_timedout(unsigned long data) +{ + struct iscsi_cmd_task *ctask = (struct iscsi_cmd_task *)data; + struct iscsi_conn *conn = ctask->conn; + struct iscsi_session *session = conn->session; + + spin_lock(&session->lock); + if (conn->tmabort_state == TMABORT_INITIAL) { + conn->tmabort_state = TMABORT_TIMEDOUT; + debug_scsi("tmabort timedout [sc %p itt 0x%x]\n", + ctask->sc, ctask->itt); + /* unblock eh_abort() */ + wake_up(&conn->ehwait); + } + spin_unlock(&session->lock); +} + +/* must be called with the mutex lock */ +static int iscsi_exec_abort_task(struct scsi_cmnd *sc, + struct iscsi_cmd_task *ctask) +{ + struct iscsi_conn *conn = ctask->conn; + struct iscsi_session *session = conn->session; + struct iscsi_tm *hdr = &conn->tmhdr; + int rc; + + /* + * ctask timed out but session is OK requests must be serialized. + */ + memset(hdr, 0, sizeof(struct iscsi_tm)); + hdr->opcode = ISCSI_OP_SCSI_TMFUNC | ISCSI_OP_IMMEDIATE; + hdr->flags = ISCSI_TM_FUNC_ABORT_TASK; + hdr->flags |= ISCSI_FLAG_CMD_FINAL; + memcpy(hdr->lun, ctask->hdr->lun, sizeof(hdr->lun)); + hdr->rtt = ctask->hdr->itt; + hdr->refcmdsn = ctask->hdr->cmdsn; + + rc = iscsi_conn_send_generic(conn, (struct iscsi_hdr *)hdr, + NULL, 0); + if (rc) { + iscsi_conn_failure(conn, ISCSI_ERR_CONN_FAILED); + debug_scsi("abort sent failure [itt 0x%x] %d", ctask->itt, rc); + return rc; + } + + debug_scsi("abort sent [itt 0x%x]\n", ctask->itt); + + spin_lock_bh(&session->lock); + ctask->mtask = (struct iscsi_mgmt_task *) + session->mgmt_cmds[(hdr->itt & ISCSI_ITT_MASK) - + ISCSI_MGMT_ITT_OFFSET]; + + if (conn->tmabort_state == TMABORT_INITIAL) { + conn->tmfcmd_pdus_cnt++; + conn->tmabort_timer.expires = 10*HZ + jiffies; + conn->tmabort_timer.function = iscsi_tmabort_timedout; + conn->tmabort_timer.data = (unsigned long)ctask; + add_timer(&conn->tmabort_timer); + debug_scsi("abort set timeout [itt 0x%x]", ctask->itt); + } + spin_unlock_bh(&session->lock); + mutex_unlock(&conn->xmitmutex); + + /* + * block eh thread until: + * + * 1) abort response + * 2) abort timeout + * 3) session is terminated or restarted or userspace has + * given up on recovery + */ + wait_event_interruptible(conn->ehwait, + sc->SCp.phase != session->age || + session->state != ISCSI_STATE_LOGGED_IN || + conn->tmabort_state != TMABORT_INITIAL || + session->recovery_failed); + if (signal_pending(current)) + flush_signals(current); + del_timer_sync(&conn->tmabort_timer); + + mutex_lock(&conn->xmitmutex); + return 0; +} + +/* + * xmit mutex and session lock must be held + */ +#define iscsi_remove_task(tasktype) \ +static struct iscsi_##tasktype * \ +iscsi_remove_##tasktype(struct kfifo *fifo, uint32_t itt) \ +{ \ + int i, nr_tasks = __kfifo_len(fifo) / sizeof(void*); \ + struct iscsi_##tasktype *task; \ + \ + debug_scsi("searching %d tasks\n", nr_tasks); \ + \ + for (i = 0; i < nr_tasks; i++) { \ + __kfifo_get(fifo, (void*)&task, sizeof(void*)); \ + debug_scsi("check task %u\n", task->itt); \ + \ + if (task->itt == itt) { \ + debug_scsi("matched task\n"); \ + break; \ + } \ + \ + __kfifo_put(fifo, (void*)&task, sizeof(void*)); \ + } \ + return NULL; \ +} + +iscsi_remove_task(mgmt_task); +iscsi_remove_task(cmd_task); + +static int iscsi_ctask_mtask_cleanup(struct iscsi_cmd_task *ctask) +{ + struct iscsi_conn *conn = ctask->conn; + struct iscsi_session *session = conn->session; + + if (!ctask->mtask) + return -EINVAL; + + if (!iscsi_remove_mgmt_task(conn->immqueue, ctask->mtask->itt)) + list_del(&ctask->mtask->running); + __kfifo_put(session->mgmtpool.queue, (void*)&ctask->mtask, + sizeof(void*)); + ctask->mtask = NULL; + return 0; +} + +/* + * session lock and xmitmutex must be held + */ +static void fail_command(struct iscsi_conn *conn, struct iscsi_cmd_task *ctask, + int err) +{ + struct scsi_cmnd *sc; + + conn->session->tt->cleanup_cmd_task(conn, ctask); + iscsi_ctask_mtask_cleanup(ctask); + + sc = ctask->sc; + if (!sc) + return; + sc->result = err; + sc->resid = sc->request_bufflen; + iscsi_complete_command(conn->session, ctask); +} + +int iscsi_eh_abort(struct scsi_cmnd *sc) +{ + struct iscsi_cmd_task *ctask = (struct iscsi_cmd_task *)sc->SCp.ptr; + struct iscsi_conn *conn = ctask->conn; + struct iscsi_session *session = conn->session; + struct iscsi_cmd_task *pending_ctask; + int rc; + + conn->eh_abort_cnt++; + debug_scsi("aborting [sc %p itt 0x%x]\n", sc, ctask->itt); + + mutex_lock(&conn->xmitmutex); + spin_lock_bh(&session->lock); + + /* + * If we are not logged in or we have started a new session + * then let the host reset code handle this + */ + if (session->state != ISCSI_STATE_LOGGED_IN || + sc->SCp.phase != session->age) + goto failed; + + /* ctask completed before time out */ + if (!ctask->sc) + goto success; + + /* what should we do here ? */ + if (conn->ctask == ctask) { + printk(KERN_INFO "sc %p itt 0x%x partially sent. Failing " + "abort\n", sc, ctask->itt); + goto failed; + } + + /* check for the easy pending cmd abort */ + pending_ctask = iscsi_remove_cmd_task(conn->xmitqueue, ctask->itt); + if (pending_ctask) { + /* iscsi_tcp queues write transfers on the xmitqueue */ + if (list_empty(&pending_ctask->running)) { + debug_scsi("found pending task\n"); + goto success; + } else + __kfifo_put(conn->xmitqueue, (void*)&pending_ctask, + sizeof(void*)); + } + + conn->tmabort_state = TMABORT_INITIAL; + + spin_unlock_bh(&session->lock); + rc = iscsi_exec_abort_task(sc, ctask); + spin_lock_bh(&session->lock); + + iscsi_ctask_mtask_cleanup(ctask); + if (rc || sc->SCp.phase != session->age || + session->state != ISCSI_STATE_LOGGED_IN) + goto failed; + + /* ctask completed before tmf abort response */ + if (!ctask->sc) { + debug_scsi("sc completed while abort in progress\n"); + goto success; + } + + if (conn->tmabort_state != TMABORT_SUCCESS) { + spin_unlock_bh(&session->lock); + iscsi_conn_failure(conn, ISCSI_ERR_CONN_FAILED); + spin_lock_bh(&session->lock); + goto failed; + } + +success: + debug_scsi("abort success [sc %lx itt 0x%x]\n", (long)sc, ctask->itt); + spin_unlock_bh(&session->lock); + + /* + * clean up task if aborted. we have the xmitmutex so grab + * the recv lock as a writer + */ + write_lock_bh(conn->recv_lock); + spin_lock(&session->lock); + fail_command(conn, ctask, DID_ABORT << 16); + spin_unlock(&session->lock); + write_unlock_bh(conn->recv_lock); + + mutex_unlock(&conn->xmitmutex); + return SUCCESS; + +failed: + spin_unlock_bh(&session->lock); + mutex_unlock(&conn->xmitmutex); + + debug_scsi("abort failed [sc %lx itt 0x%x]\n", (long)sc, ctask->itt); + return FAILED; +} +EXPORT_SYMBOL_GPL(iscsi_eh_abort); + +int +iscsi_pool_init(struct iscsi_queue *q, int max, void ***items, int item_size) +{ + int i; + + *items = kmalloc(max * sizeof(void*), GFP_KERNEL); + if (*items == NULL) + return -ENOMEM; + + q->max = max; + q->pool = kmalloc(max * sizeof(void*), GFP_KERNEL); + if (q->pool == NULL) { + kfree(*items); + return -ENOMEM; + } + + q->queue = kfifo_init((void*)q->pool, max * sizeof(void*), + GFP_KERNEL, NULL); + if (q->queue == ERR_PTR(-ENOMEM)) { + kfree(q->pool); + kfree(*items); + return -ENOMEM; + } + + for (i = 0; i < max; i++) { + q->pool[i] = kmalloc(item_size, GFP_KERNEL); + if (q->pool[i] == NULL) { + int j; + + for (j = 0; j < i; j++) + kfree(q->pool[j]); + + kfifo_free(q->queue); + kfree(q->pool); + kfree(*items); + return -ENOMEM; + } + memset(q->pool[i], 0, item_size); + (*items)[i] = q->pool[i]; + __kfifo_put(q->queue, (void*)&q->pool[i], sizeof(void*)); + } + return 0; +} +EXPORT_SYMBOL_GPL(iscsi_pool_init); + +void iscsi_pool_free(struct iscsi_queue *q, void **items) +{ + int i; + + for (i = 0; i < q->max; i++) + kfree(items[i]); + kfree(q->pool); + kfree(items); +} +EXPORT_SYMBOL_GPL(iscsi_pool_free); + +/* + * iSCSI Session's hostdata organization: + * + * *------------------* <== hostdata_session(host->hostdata) + * | ptr to class sess| + * |------------------| <== iscsi_hostdata(host->hostdata) + * | iscsi_session | + * *------------------* + */ + +#define hostdata_privsize(_sz) (sizeof(unsigned long) + _sz + \ + _sz % sizeof(unsigned long)) + +#define hostdata_session(_hostdata) (iscsi_ptr(*(unsigned long *)_hostdata)) + +/** + * iscsi_session_setup - create iscsi cls session and host and session + * @scsit: scsi transport template + * @iscsit: iscsi transport template + * @initial_cmdsn: initial CmdSN + * @hostno: host no allocated + * + * This can be used by software iscsi_transports that allocate + * a session per scsi host. + **/ +struct iscsi_cls_session * +iscsi_session_setup(struct iscsi_transport *iscsit, + struct scsi_transport_template *scsit, + int cmd_task_size, int mgmt_task_size, + uint32_t initial_cmdsn, uint32_t *hostno) +{ + struct Scsi_Host *shost; + struct iscsi_session *session; + struct iscsi_cls_session *cls_session; + int cmd_i; + + shost = scsi_host_alloc(iscsit->host_template, + hostdata_privsize(sizeof(*session))); + if (!shost) + return NULL; + + shost->max_id = 1; + shost->max_channel = 0; + shost->max_lun = iscsit->max_lun; + shost->max_cmd_len = iscsit->max_cmd_len; + shost->transportt = scsit; + shost->transportt->create_work_queue = 1; + *hostno = shost->host_no; + + session = iscsi_hostdata(shost->hostdata); + memset(session, 0, sizeof(struct iscsi_session)); + session->host = shost; + session->state = ISCSI_STATE_FREE; + session->mgmtpool_max = ISCSI_MGMT_CMDS_MAX; + session->cmds_max = ISCSI_XMIT_CMDS_MAX; + session->cmdsn = initial_cmdsn; + session->exp_cmdsn = initial_cmdsn + 1; + session->max_cmdsn = initial_cmdsn + 1; + session->max_r2t = 1; + session->tt = iscsit; + + /* initialize SCSI PDU commands pool */ + if (iscsi_pool_init(&session->cmdpool, session->cmds_max, + (void***)&session->cmds, + cmd_task_size + sizeof(struct iscsi_cmd_task))) + goto cmdpool_alloc_fail; + + /* pre-format cmds pool with ITT */ + for (cmd_i = 0; cmd_i < session->cmds_max; cmd_i++) { + struct iscsi_cmd_task *ctask = session->cmds[cmd_i]; + + if (cmd_task_size) + ctask->dd_data = &ctask[1]; + ctask->itt = cmd_i; + } + + spin_lock_init(&session->lock); + INIT_LIST_HEAD(&session->connections); + + /* initialize immediate command pool */ + if (iscsi_pool_init(&session->mgmtpool, session->mgmtpool_max, + (void***)&session->mgmt_cmds, + mgmt_task_size + sizeof(struct iscsi_mgmt_task))) + goto mgmtpool_alloc_fail; + + + /* pre-format immediate cmds pool with ITT */ + for (cmd_i = 0; cmd_i < session->mgmtpool_max; cmd_i++) { + struct iscsi_mgmt_task *mtask = session->mgmt_cmds[cmd_i]; + + if (mgmt_task_size) + mtask->dd_data = &mtask[1]; + mtask->itt = ISCSI_MGMT_ITT_OFFSET + cmd_i; + mtask->data = kmalloc(DEFAULT_MAX_RECV_DATA_SEGMENT_LENGTH, + GFP_KERNEL); + if (!mtask->data) { + int j; + + for (j = 0; j < cmd_i; j++) + kfree(session->mgmt_cmds[j]->data); + goto immdata_alloc_fail; + } + } + + if (scsi_add_host(shost, NULL)) + goto add_host_fail; + + cls_session = iscsi_create_session(shost, iscsit, 0); + if (!cls_session) + goto cls_session_fail; + *(unsigned long*)shost->hostdata = (unsigned long)cls_session; + + return cls_session; + +cls_session_fail: + scsi_remove_host(shost); +add_host_fail: + for (cmd_i = 0; cmd_i < session->mgmtpool_max; cmd_i++) + kfree(session->mgmt_cmds[cmd_i]->data); +immdata_alloc_fail: + iscsi_pool_free(&session->mgmtpool, (void**)session->mgmt_cmds); +mgmtpool_alloc_fail: + iscsi_pool_free(&session->cmdpool, (void**)session->cmds); +cmdpool_alloc_fail: + scsi_host_put(shost); + return NULL; +} +EXPORT_SYMBOL_GPL(iscsi_session_setup); + +/** + * iscsi_session_teardown - destroy session, host, and cls_session + * shost: scsi host + * + * This can be used by software iscsi_transports that allocate + * a session per scsi host. + **/ +void iscsi_session_teardown(struct iscsi_cls_session *cls_session) +{ + struct Scsi_Host *shost = iscsi_session_to_shost(cls_session); + struct iscsi_session *session = iscsi_hostdata(shost->hostdata); + int cmd_i; + + scsi_remove_host(shost); + + for (cmd_i = 0; cmd_i < session->mgmtpool_max; cmd_i++) + kfree(session->mgmt_cmds[cmd_i]->data); + + iscsi_pool_free(&session->mgmtpool, (void**)session->mgmt_cmds); + iscsi_pool_free(&session->cmdpool, (void**)session->cmds); + + iscsi_destroy_session(cls_session); + scsi_host_put(shost); +} +EXPORT_SYMBOL_GPL(iscsi_session_teardown); + +/** + * iscsi_conn_setup - create iscsi_cls_conn and iscsi_conn + * @cls_session: iscsi_cls_session + * @conn_idx: cid + **/ +struct iscsi_cls_conn * +iscsi_conn_setup(struct iscsi_cls_session *cls_session, uint32_t conn_idx) +{ + struct iscsi_session *session = class_to_transport_session(cls_session); + struct iscsi_conn *conn; + struct iscsi_cls_conn *cls_conn; + + cls_conn = iscsi_create_conn(cls_session, conn_idx); + if (!cls_conn) + return NULL; + conn = cls_conn->dd_data; + memset(conn, 0, sizeof(*conn)); + + conn->session = session; + conn->cls_conn = cls_conn; + conn->c_stage = ISCSI_CONN_INITIAL_STAGE; + conn->id = conn_idx; + conn->exp_statsn = 0; + conn->tmabort_state = TMABORT_INITIAL; + INIT_LIST_HEAD(&conn->run_list); + INIT_LIST_HEAD(&conn->mgmt_run_list); + + /* initialize general xmit PDU commands queue */ + conn->xmitqueue = kfifo_alloc(session->cmds_max * sizeof(void*), + GFP_KERNEL, NULL); + if (conn->xmitqueue == ERR_PTR(-ENOMEM)) + goto xmitqueue_alloc_fail; + + /* initialize general immediate & non-immediate PDU commands queue */ + conn->immqueue = kfifo_alloc(session->mgmtpool_max * sizeof(void*), + GFP_KERNEL, NULL); + if (conn->immqueue == ERR_PTR(-ENOMEM)) + goto immqueue_alloc_fail; + + conn->mgmtqueue = kfifo_alloc(session->mgmtpool_max * sizeof(void*), + GFP_KERNEL, NULL); + if (conn->mgmtqueue == ERR_PTR(-ENOMEM)) + goto mgmtqueue_alloc_fail; + + INIT_WORK(&conn->xmitwork, iscsi_xmitworker, conn); + + /* allocate login_mtask used for the login/text sequences */ + spin_lock_bh(&session->lock); + if (!__kfifo_get(session->mgmtpool.queue, + (void*)&conn->login_mtask, + sizeof(void*))) { + spin_unlock_bh(&session->lock); + goto login_mtask_alloc_fail; + } + spin_unlock_bh(&session->lock); + + init_timer(&conn->tmabort_timer); + mutex_init(&conn->xmitmutex); + init_waitqueue_head(&conn->ehwait); + + return cls_conn; + +login_mtask_alloc_fail: + kfifo_free(conn->mgmtqueue); +mgmtqueue_alloc_fail: + kfifo_free(conn->immqueue); +immqueue_alloc_fail: + kfifo_free(conn->xmitqueue); +xmitqueue_alloc_fail: + iscsi_destroy_conn(cls_conn); + return NULL; +} +EXPORT_SYMBOL_GPL(iscsi_conn_setup); + +/** + * iscsi_conn_teardown - teardown iscsi connection + * cls_conn: iscsi class connection + * + * TODO: we may need to make this into a two step process + * like scsi-mls remove + put host + */ +void iscsi_conn_teardown(struct iscsi_cls_conn *cls_conn) +{ + struct iscsi_conn *conn = cls_conn->dd_data; + struct iscsi_session *session = conn->session; + unsigned long flags; + + mutex_lock(&conn->xmitmutex); + set_bit(ISCSI_SUSPEND_BIT, &conn->suspend_tx); + if (conn->c_stage == ISCSI_CONN_INITIAL_STAGE) { + if (session->tt->suspend_conn_recv) + session->tt->suspend_conn_recv(conn); + + session->tt->terminate_conn(conn); + } + + spin_lock_bh(&session->lock); + conn->c_stage = ISCSI_CONN_CLEANUP_WAIT; + if (session->leadconn == conn) { + /* + * leading connection? then give up on recovery. + */ + session->state = ISCSI_STATE_TERMINATE; + wake_up(&conn->ehwait); + } + spin_unlock_bh(&session->lock); + + mutex_unlock(&conn->xmitmutex); + + /* + * Block until all in-progress commands for this connection + * time out or fail. + */ + for (;;) { + spin_lock_irqsave(session->host->host_lock, flags); + if (!session->host->host_busy) { /* OK for ERL == 0 */ + spin_unlock_irqrestore(session->host->host_lock, flags); + break; + } + spin_unlock_irqrestore(session->host->host_lock, flags); + msleep_interruptible(500); + printk("conn_destroy(): host_busy %d host_failed %d\n", + session->host->host_busy, session->host->host_failed); + /* + * force eh_abort() to unblock + */ + wake_up(&conn->ehwait); + } + + spin_lock_bh(&session->lock); + __kfifo_put(session->mgmtpool.queue, (void*)&conn->login_mtask, + sizeof(void*)); + list_del(&conn->item); + if (list_empty(&session->connections)) + session->leadconn = NULL; + if (session->leadconn && session->leadconn == conn) + session->leadconn = container_of(session->connections.next, + struct iscsi_conn, item); + + if (session->leadconn == NULL) + /* no connections exits.. reset sequencing */ + session->cmdsn = session->max_cmdsn = session->exp_cmdsn = 1; + spin_unlock_bh(&session->lock); + + kfifo_free(conn->xmitqueue); + kfifo_free(conn->immqueue); + kfifo_free(conn->mgmtqueue); + + iscsi_destroy_conn(cls_conn); +} +EXPORT_SYMBOL_GPL(iscsi_conn_teardown); + +int iscsi_conn_start(struct iscsi_cls_conn *cls_conn) +{ + struct iscsi_conn *conn = cls_conn->dd_data; + struct iscsi_session *session = conn->session; + + if (session == NULL) { + printk(KERN_ERR "iscsi: can't start unbound connection\n"); + return -EPERM; + } + + spin_lock_bh(&session->lock); + conn->c_stage = ISCSI_CONN_STARTED; + session->state = ISCSI_STATE_LOGGED_IN; + + switch(conn->stop_stage) { + case STOP_CONN_RECOVER: + /* + * unblock eh_abort() if it is blocked. re-try all + * commands after successful recovery + */ + session->conn_cnt++; + conn->stop_stage = 0; + conn->tmabort_state = TMABORT_INITIAL; + session->age++; + session->recovery_failed = 0; + spin_unlock_bh(&session->lock); + + iscsi_unblock_session(session_to_cls(session)); + wake_up(&conn->ehwait); + return 0; + case STOP_CONN_TERM: + session->conn_cnt++; + conn->stop_stage = 0; + break; + case STOP_CONN_SUSPEND: + conn->stop_stage = 0; + clear_bit(ISCSI_SUSPEND_BIT, &conn->suspend_rx); + clear_bit(ISCSI_SUSPEND_BIT, &conn->suspend_tx); + break; + default: + break; + } + spin_unlock_bh(&session->lock); + + return 0; +} +EXPORT_SYMBOL_GPL(iscsi_conn_start); + +static void +flush_control_queues(struct iscsi_session *session, struct iscsi_conn *conn) +{ + struct iscsi_mgmt_task *mtask, *tmp; + + /* handle pending */ + while (__kfifo_get(conn->immqueue, (void*)&mtask, sizeof(void*)) || + __kfifo_get(conn->mgmtqueue, (void*)&mtask, sizeof(void*))) { + if (mtask == conn->login_mtask) + continue; + debug_scsi("flushing pending mgmt task itt 0x%x\n", mtask->itt); + __kfifo_put(session->mgmtpool.queue, (void*)&mtask, + sizeof(void*)); + } + + /* handle running */ + list_for_each_entry_safe(mtask, tmp, &conn->mgmt_run_list, running) { + debug_scsi("flushing running mgmt task itt 0x%x\n", mtask->itt); + if (mtask == conn->login_mtask) + continue; + list_del(&mtask->running); + __kfifo_put(session->mgmtpool.queue, (void*)&conn->mtask, + sizeof(void*)); + } + + conn->mtask = NULL; +} + +/* Fail commands. Mutex and session lock held and recv side suspended */ +static void fail_all_commands(struct iscsi_conn *conn) +{ + struct iscsi_cmd_task *ctask, *tmp; + + /* flush pending */ + while (__kfifo_get(conn->xmitqueue, (void*)&ctask, sizeof(void*))) { + debug_scsi("failing pending sc %p itt 0x%x\n", ctask->sc, + ctask->itt); + fail_command(conn, ctask, DID_BUS_BUSY << 16); + } + + /* fail all other running */ + list_for_each_entry_safe(ctask, tmp, &conn->run_list, running) { + debug_scsi("failing in progress sc %p itt 0x%x\n", + ctask->sc, ctask->itt); + fail_command(conn, ctask, DID_BUS_BUSY << 16); + } + + conn->ctask = NULL; +} + +void iscsi_start_session_recovery(struct iscsi_session *session, + struct iscsi_conn *conn, int flag) +{ + spin_lock_bh(&session->lock); + if (conn->stop_stage == STOP_CONN_RECOVER || + conn->stop_stage == STOP_CONN_TERM) { + spin_unlock_bh(&session->lock); + return; + } + conn->stop_stage = flag; + spin_unlock_bh(&session->lock); + + if (session->tt->suspend_conn_recv) + session->tt->suspend_conn_recv(conn); + + mutex_lock(&conn->xmitmutex); + spin_lock_bh(&session->lock); + conn->c_stage = ISCSI_CONN_STOPPED; + set_bit(ISCSI_SUSPEND_BIT, &conn->suspend_tx); + + session->conn_cnt--; + if (session->conn_cnt == 0 || session->leadconn == conn) + session->state = ISCSI_STATE_FAILED; + + spin_unlock_bh(&session->lock); + + session->tt->terminate_conn(conn); + /* + * flush queues. + */ + spin_lock_bh(&session->lock); + fail_all_commands(conn); + flush_control_queues(session, conn); + spin_unlock_bh(&session->lock); + + /* + * for connection level recovery we should not calculate + * header digest. conn->hdr_size used for optimization + * in hdr_extract() and will be re-negotiated at + * set_param() time. + */ + if (flag == STOP_CONN_RECOVER) { + conn->hdrdgst_en = 0; + conn->datadgst_en = 0; + + if (session->state == ISCSI_STATE_FAILED) + iscsi_block_session(session_to_cls(session)); + } + mutex_unlock(&conn->xmitmutex); +} +EXPORT_SYMBOL_GPL(iscsi_start_session_recovery); + +void iscsi_conn_stop(struct iscsi_cls_conn *cls_conn, int flag) +{ + struct iscsi_conn *conn = cls_conn->dd_data; + struct iscsi_session *session = conn->session; + + switch (flag) { + case STOP_CONN_RECOVER: + case STOP_CONN_TERM: + iscsi_start_session_recovery(session, conn, flag); + return; + case STOP_CONN_SUSPEND: + if (session->tt->suspend_conn_recv) + session->tt->suspend_conn_recv(conn); + + mutex_lock(&conn->xmitmutex); + spin_lock_bh(&session->lock); + + conn->stop_stage = flag; + conn->c_stage = ISCSI_CONN_STOPPED; + set_bit(ISCSI_SUSPEND_BIT, &conn->suspend_tx); + + spin_unlock_bh(&session->lock); + mutex_unlock(&conn->xmitmutex); + break; + default: + printk(KERN_ERR "invalid stop flag %d\n", flag); + } +} +EXPORT_SYMBOL_GPL(iscsi_conn_stop); + +int iscsi_conn_bind(struct iscsi_cls_session *cls_session, + struct iscsi_cls_conn *cls_conn, int is_leading) +{ + struct iscsi_session *session = class_to_transport_session(cls_session); + struct iscsi_conn *tmp = ERR_PTR(-EEXIST), *conn = cls_conn->dd_data; + + /* lookup for existing connection */ + spin_lock_bh(&session->lock); + list_for_each_entry(tmp, &session->connections, item) { + if (tmp == conn) { + if (conn->c_stage != ISCSI_CONN_STOPPED || + conn->stop_stage == STOP_CONN_TERM) { + printk(KERN_ERR "iscsi_tcp: can't bind " + "non-stopped connection (%d:%d)\n", + conn->c_stage, conn->stop_stage); + spin_unlock_bh(&session->lock); + return -EIO; + } + break; + } + } + if (tmp != conn) { + /* bind new iSCSI connection to session */ + conn->session = session; + list_add(&conn->item, &session->connections); + } + spin_unlock_bh(&session->lock); + + if (is_leading) + session->leadconn = conn; + + /* + * Unblock xmitworker(), Login Phase will pass through. + */ + clear_bit(ISCSI_SUSPEND_BIT, &conn->suspend_rx); + clear_bit(ISCSI_SUSPEND_BIT, &conn->suspend_tx); + return 0; +} +EXPORT_SYMBOL_GPL(iscsi_conn_bind); + +MODULE_AUTHOR("Mike Christie"); +MODULE_DESCRIPTION("iSCSI library functions"); +MODULE_LICENSE("GPL"); diff -puN drivers/scsi/Makefile~git-scsi-misc drivers/scsi/Makefile --- devel/drivers/scsi/Makefile~git-scsi-misc 2006-04-30 00:30:24.000000000 -0700 +++ devel-akpm/drivers/scsi/Makefile 2006-04-30 00:30:24.000000000 -0700 @@ -34,7 +34,7 @@ obj-$(CONFIG_SCSI_ISCSI_ATTRS) += scsi_t obj-$(CONFIG_SCSI_SAS_ATTRS) += scsi_transport_sas.o obj-$(CONFIG_SCSI_SAS_DOMAIN_ATTRS) += sas/ -obj-$(CONFIG_ISCSI_TCP) += iscsi_tcp.o +obj-$(CONFIG_ISCSI_TCP) += libiscsi.o iscsi_tcp.o obj-$(CONFIG_SCSI_AMIGA7XX) += amiga7xx.o 53c7xx.o obj-$(CONFIG_A3000_SCSI) += a3000.o wd33c93.o obj-$(CONFIG_A2091_SCSI) += a2091.o wd33c93.o diff -puN drivers/scsi/scsi_devinfo.c~git-scsi-misc drivers/scsi/scsi_devinfo.c --- devel/drivers/scsi/scsi_devinfo.c~git-scsi-misc 2006-04-30 00:30:24.000000000 -0700 +++ devel-akpm/drivers/scsi/scsi_devinfo.c 2006-04-30 00:30:24.000000000 -0700 @@ -157,6 +157,8 @@ static struct { {"HITACHI", "DF400", "*", BLIST_SPARSELUN}, {"HITACHI", "DF500", "*", BLIST_SPARSELUN}, {"HITACHI", "DF600", "*", BLIST_SPARSELUN}, + {"HITACHI", "DISK-SUBSYSTEM", "*", BLIST_ATTACH_PQ3 | BLIST_SPARSELUN | BLIST_LARGELUN}, + {"HITACHI", "OPEN-E", "*", BLIST_ATTACH_PQ3 | BLIST_SPARSELUN | BLIST_LARGELUN}, {"HP", "A6189A", NULL, BLIST_SPARSELUN | BLIST_LARGELUN}, /* HP VA7400 */ {"HP", "OPEN-", "*", BLIST_SPARSELUN | BLIST_LARGELUN}, /* HP XP Arrays */ {"HP", "NetRAID-4M", NULL, BLIST_FORCELUN}, diff -puN drivers/scsi/scsi_lib.c~git-scsi-misc drivers/scsi/scsi_lib.c --- devel/drivers/scsi/scsi_lib.c~git-scsi-misc 2006-04-30 00:30:24.000000000 -0700 +++ devel-akpm/drivers/scsi/scsi_lib.c 2006-04-30 00:30:24.000000000 -0700 @@ -2350,3 +2350,61 @@ scsi_target_unblock(struct device *dev) device_for_each_child(dev, NULL, target_unblock); } EXPORT_SYMBOL_GPL(scsi_target_unblock); + +/** + * scsi_kmap_atomic_sg - find and atomically map an sg-elemnt + * @sg: scatter-gather list + * @sg_count: number of segments in sg + * @offset: offset in bytes into sg, on return offset into the mapped area + * @len: bytes to map, on return number of bytes mapped + * + * Returns virtual address of the start of the mapped page + */ +void *scsi_kmap_atomic_sg(struct scatterlist *sg, int sg_count, + size_t *offset, size_t *len) +{ + int i; + size_t sg_len = 0, len_complete = 0; + struct page *page; + + for (i = 0; i < sg_count; i++) { + len_complete = sg_len; /* Complete sg-entries */ + sg_len += sg[i].length; + if (sg_len > *offset) + break; + } + + if (unlikely(i == sg_count)) { + printk(KERN_ERR "%s: Bytes in sg: %zu, requested offset %zu, " + "elements %d\n", + __FUNCTION__, sg_len, *offset, sg_count); + WARN_ON(1); + return NULL; + } + + /* Offset starting from the beginning of first page in this sg-entry */ + *offset = *offset - len_complete + sg[i].offset; + + /* Assumption: contiguous pages can be accessed as "page + i" */ + page = nth_page(sg[i].page, (*offset >> PAGE_SHIFT)); + *offset &= ~PAGE_MASK; + + /* Bytes in this sg-entry from *offset to the end of the page */ + sg_len = PAGE_SIZE - *offset; + if (*len > sg_len) + *len = sg_len; + + return kmap_atomic(page, KM_BIO_SRC_IRQ); +} +EXPORT_SYMBOL(scsi_kmap_atomic_sg); + +/** + * scsi_kunmap_atomic_sg - atomically unmap a virtual address, previously + * mapped with scsi_kmap_atomic_sg + * @virt: virtual address to be unmapped + */ +void scsi_kunmap_atomic_sg(void *virt) +{ + kunmap_atomic(virt, KM_BIO_SRC_IRQ); +} +EXPORT_SYMBOL(scsi_kunmap_atomic_sg); diff -puN drivers/scsi/scsi_logging.h~git-scsi-misc drivers/scsi/scsi_logging.h --- devel/drivers/scsi/scsi_logging.h~git-scsi-misc 2006-04-30 00:30:24.000000000 -0700 +++ devel-akpm/drivers/scsi/scsi_logging.h 2006-04-30 00:30:24.000000000 -0700 @@ -45,10 +45,12 @@ extern unsigned int scsi_logging_level; ((scsi_logging_level >> (SHIFT)) & ((1 << (BITS)) - 1)) #define SCSI_CHECK_LOGGING(SHIFT, BITS, LEVEL, CMD) \ -{ \ +do { \ if (unlikely((SCSI_LOG_LEVEL(SHIFT, BITS)) > (LEVEL))) \ - (CMD); \ -} + do { \ + CMD; \ + } while (0); \ +} while (0) #else #define SCSI_CHECK_LOGGING(SHIFT, BITS, LEVEL, CMD) #endif /* CONFIG_SCSI_LOGGING */ diff -puN drivers/scsi/scsi_scan.c~git-scsi-misc drivers/scsi/scsi_scan.c --- devel/drivers/scsi/scsi_scan.c~git-scsi-misc 2006-04-30 00:30:24.000000000 -0700 +++ devel-akpm/drivers/scsi/scsi_scan.c 2006-04-30 00:30:24.000000000 -0700 @@ -816,6 +816,32 @@ static inline void scsi_destroy_sdev(str put_device(&sdev->sdev_gendev); } +#ifdef CONFIG_SCSI_LOGGING +/** + * scsi_inq_str - print INQUIRY data from min to max index, + * strip trailing whitespace + * @buf: Output buffer with at least end-first+1 bytes of space + * @inq: Inquiry buffer (input) + * @first: Offset of string into inq + * @end: Index after last character in inq + */ +static unsigned char *scsi_inq_str(unsigned char *buf, unsigned char *inq, + unsigned first, unsigned end) +{ + unsigned term = 0, idx; + + for (idx = 0; idx + first < end && idx + first < inq[4] + 5; idx++) { + if (inq[idx+first] > ' ') { + buf[idx] = inq[idx+first]; + term = idx+1; + } else { + buf[idx] = ' '; + } + } + buf[term] = 0; + return buf; +} +#endif /** * scsi_probe_and_add_lun - probe a LUN, if a LUN is found add it @@ -880,10 +906,12 @@ static int scsi_probe_and_add_lun(struct if (scsi_probe_lun(sdev, result, result_len, &bflags)) goto out_free_result; + if (bflagsp) + *bflagsp = bflags; /* * result contains valid SCSI INQUIRY data. */ - if ((result[0] >> 5) == 3) { + if (((result[0] >> 5) == 3) && !(bflags & BLIST_ATTACH_PQ3)) { /* * For a Peripheral qualifier 3 (011b), the SCSI * spec says: The device server is not capable of @@ -894,9 +922,22 @@ static int scsi_probe_and_add_lun(struct * logical disk configured at sdev->lun, but there * is a target id responding. */ - SCSI_LOG_SCAN_BUS(3, printk(KERN_INFO - "scsi scan: peripheral qualifier of 3," - " no device added\n")); + SCSI_LOG_SCAN_BUS(2, sdev_printk(KERN_INFO, sdev, "scsi scan:" + " peripheral qualifier of 3, device not" + " added\n")) + if (lun == 0) { + SCSI_LOG_SCAN_BUS(1, { + unsigned char vend[9]; + unsigned char mod[17]; + + sdev_printk(KERN_INFO, sdev, + "scsi scan: consider passing scsi_mod." + "dev_flags=%s:%s:0x240 or 0x800240\n", + scsi_inq_str(vend, result, 8, 16), + scsi_inq_str(mod, result, 16, 32)); + }); + } + res = SCSI_SCAN_TARGET_PRESENT; goto out_free_result; } @@ -920,8 +961,6 @@ static int scsi_probe_and_add_lun(struct sdev->lockable = 0; scsi_unlock_floptical(sdev, result); } - if (bflagsp) - *bflagsp = bflags; } out_free_result: @@ -946,7 +985,6 @@ static int scsi_probe_and_add_lun(struct * scsi_sequential_lun_scan - sequentially scan a SCSI target * @starget: pointer to target structure to scan * @bflags: black/white list flag for LUN 0 - * @lun0_res: result of scanning LUN 0 * * Description: * Generally, scan from LUN 1 (LUN 0 is assumed to already have been @@ -956,8 +994,7 @@ static int scsi_probe_and_add_lun(struct * Modifies sdevscan->lun. **/ static void scsi_sequential_lun_scan(struct scsi_target *starget, - int bflags, int lun0_res, int scsi_level, - int rescan) + int bflags, int scsi_level, int rescan) { unsigned int sparse_lun, lun, max_dev_lun; struct Scsi_Host *shost = dev_to_shost(starget->dev.parent); @@ -978,13 +1015,6 @@ static void scsi_sequential_lun_scan(str sparse_lun = 0; /* - * If not sparse lun and no device attached at LUN 0 do not scan - * any further. - */ - if (!sparse_lun && (lun0_res != SCSI_SCAN_LUN_PRESENT)) - return; - - /* * If less than SCSI_1_CSS, and no special lun scaning, stop * scanning; this matches 2.4 behaviour, but could just be a bug * (to continue scanning a SCSI_1_CSS device). @@ -1395,7 +1425,7 @@ static void __scsi_scan_target(struct de * do a sequential scan. */ scsi_sequential_lun_scan(starget, bflags, - res, starget->scsi_level, rescan); + starget->scsi_level, rescan); } out_reap: diff -puN drivers/scsi/scsi_transport_iscsi.c~git-scsi-misc drivers/scsi/scsi_transport_iscsi.c --- devel/drivers/scsi/scsi_transport_iscsi.c~git-scsi-misc 2006-04-30 00:30:24.000000000 -0700 +++ devel-akpm/drivers/scsi/scsi_transport_iscsi.c 2006-04-30 00:30:24.000000000 -0700 @@ -31,31 +31,25 @@ #include #include -#define ISCSI_SESSION_ATTRS 8 -#define ISCSI_CONN_ATTRS 6 +#define ISCSI_SESSION_ATTRS 11 +#define ISCSI_CONN_ATTRS 10 +#define ISCSI_HOST_ATTRS 0 struct iscsi_internal { struct scsi_transport_template t; struct iscsi_transport *iscsi_transport; struct list_head list; - /* - * based on transport capabilities, at register time we set these - * bits to tell the transport class it wants attributes displayed - * in sysfs or that it can support different iSCSI Data-Path - * capabilities - */ - uint32_t param_mask; - struct class_device cdev; - /* - * We do not have any private or other attrs. - */ + + struct class_device_attribute *host_attrs[ISCSI_HOST_ATTRS + 1]; struct transport_container conn_cont; struct class_device_attribute *conn_attrs[ISCSI_CONN_ATTRS + 1]; struct transport_container session_cont; struct class_device_attribute *session_attrs[ISCSI_SESSION_ATTRS + 1]; }; +static int iscsi_session_nr; /* sysfs session id for next new session */ + /* * list of registered transports and lock that must * be held while accessing list. The iscsi_transport_lock must @@ -120,6 +114,24 @@ static struct attribute_group iscsi_tran .attrs = iscsi_transport_attrs, }; +static int iscsi_setup_host(struct transport_container *tc, struct device *dev, + struct class_device *cdev) +{ + struct Scsi_Host *shost = dev_to_shost(dev); + struct iscsi_host *ihost = shost->shost_data; + + memset(ihost, 0, sizeof(*ihost)); + INIT_LIST_HEAD(&ihost->sessions); + mutex_init(&ihost->mutex); + return 0; +} + +static DECLARE_TRANSPORT_CLASS(iscsi_host_class, + "iscsi_host", + iscsi_setup_host, + NULL, + NULL); + static DECLARE_TRANSPORT_CLASS(iscsi_session_class, "iscsi_session", NULL, @@ -165,14 +177,23 @@ static DEFINE_SPINLOCK(sesslock); static LIST_HEAD(connlist); static DEFINE_SPINLOCK(connlock); -static struct iscsi_cls_session *iscsi_session_lookup(uint64_t handle) +static uint32_t iscsi_conn_get_sid(struct iscsi_cls_conn *conn) +{ + struct iscsi_cls_session *sess = iscsi_dev_to_session(conn->dev.parent); + return sess->sid; +} + +/* + * Returns the matching session to a given sid + */ +static struct iscsi_cls_session *iscsi_session_lookup(uint32_t sid) { unsigned long flags; struct iscsi_cls_session *sess; spin_lock_irqsave(&sesslock, flags); list_for_each_entry(sess, &sesslist, sess_list) { - if (sess == iscsi_ptr(handle)) { + if (sess->sid == sid) { spin_unlock_irqrestore(&sesslock, flags); return sess; } @@ -181,14 +202,17 @@ static struct iscsi_cls_session *iscsi_s return NULL; } -static struct iscsi_cls_conn *iscsi_conn_lookup(uint64_t handle) +/* + * Returns the matching connection to a given sid / cid tuple + */ +static struct iscsi_cls_conn *iscsi_conn_lookup(uint32_t sid, uint32_t cid) { unsigned long flags; struct iscsi_cls_conn *conn; spin_lock_irqsave(&connlock, flags); list_for_each_entry(conn, &connlist, conn_list) { - if (conn == iscsi_ptr(handle)) { + if ((conn->cid == cid) && (iscsi_conn_get_sid(conn) == sid)) { spin_unlock_irqrestore(&connlock, flags); return conn; } @@ -209,6 +233,7 @@ static void iscsi_session_release(struct shost = iscsi_session_to_shost(session); scsi_host_put(shost); + kfree(session->targetname); kfree(session); module_put(transport->owner); } @@ -218,30 +243,95 @@ static int iscsi_is_session_dev(const st return dev->release == iscsi_session_release; } +static int iscsi_user_scan(struct Scsi_Host *shost, uint channel, + uint id, uint lun) +{ + struct iscsi_host *ihost = shost->shost_data; + struct iscsi_cls_session *session; + + mutex_lock(&ihost->mutex); + list_for_each_entry(session, &ihost->sessions, host_list) { + if ((channel == SCAN_WILD_CARD || + channel == session->channel) && + (id == SCAN_WILD_CARD || id == session->target_id)) + scsi_scan_target(&session->dev, session->channel, + session->target_id, lun, 1); + } + mutex_unlock(&ihost->mutex); + + return 0; +} + +static void session_recovery_timedout(void *data) +{ + struct iscsi_cls_session *session = data; + + dev_printk(KERN_INFO, &session->dev, "session recovery timed out " + "after %d secs\n", session->recovery_tmo); + + if (session->transport->session_recovery_timedout) + session->transport->session_recovery_timedout(session); + + scsi_target_unblock(&session->dev); +} + +void iscsi_unblock_session(struct iscsi_cls_session *session) +{ + if (!cancel_delayed_work(&session->recovery_work)) + flush_scheduled_work(); + scsi_target_unblock(&session->dev); +} +EXPORT_SYMBOL_GPL(iscsi_unblock_session); + +void iscsi_block_session(struct iscsi_cls_session *session) +{ + scsi_target_block(&session->dev); + schedule_delayed_work(&session->recovery_work, + session->recovery_tmo * HZ); +} +EXPORT_SYMBOL_GPL(iscsi_block_session); + /** * iscsi_create_session - create iscsi class session * @shost: scsi host * @transport: iscsi transport * - * This can be called from a LLD or iscsi_transport + * This can be called from a LLD or iscsi_transport. **/ struct iscsi_cls_session * -iscsi_create_session(struct Scsi_Host *shost, struct iscsi_transport *transport) +iscsi_create_session(struct Scsi_Host *shost, + struct iscsi_transport *transport, int channel) { + struct iscsi_host *ihost; struct iscsi_cls_session *session; int err; if (!try_module_get(transport->owner)) return NULL; - session = kzalloc(sizeof(*session), GFP_KERNEL); + session = kzalloc(sizeof(*session) + transport->sessiondata_size, + GFP_KERNEL); if (!session) goto module_put; session->transport = transport; + session->recovery_tmo = 120; + INIT_WORK(&session->recovery_work, session_recovery_timedout, session); + INIT_LIST_HEAD(&session->host_list); + INIT_LIST_HEAD(&session->sess_list); + + if (transport->sessiondata_size) + session->dd_data = &session[1]; /* this is released in the dev's release function */ scsi_host_get(shost); - snprintf(session->dev.bus_id, BUS_ID_SIZE, "session%u", shost->host_no); + ihost = shost->shost_data; + + session->sid = iscsi_session_nr++; + session->channel = channel; + session->target_id = ihost->next_target_id++; + + snprintf(session->dev.bus_id, BUS_ID_SIZE, "session%u", + session->sid); session->dev.parent = &shost->shost_gendev; session->dev.release = iscsi_session_release; err = device_register(&session->dev); @@ -252,6 +342,10 @@ iscsi_create_session(struct Scsi_Host *s } transport_register_device(&session->dev); + mutex_lock(&ihost->mutex); + list_add(&session->host_list, &ihost->sessions); + mutex_unlock(&ihost->mutex); + return session; free_session: @@ -272,6 +366,16 @@ EXPORT_SYMBOL_GPL(iscsi_create_session); **/ int iscsi_destroy_session(struct iscsi_cls_session *session) { + struct Scsi_Host *shost = iscsi_session_to_shost(session); + struct iscsi_host *ihost = shost->shost_data; + + if (!cancel_delayed_work(&session->recovery_work)) + flush_scheduled_work(); + + mutex_lock(&ihost->mutex); + list_del(&session->host_list); + mutex_unlock(&ihost->mutex); + transport_unregister_device(&session->dev); device_unregister(&session->dev); return 0; @@ -284,6 +388,7 @@ static void iscsi_conn_release(struct de struct iscsi_cls_conn *conn = iscsi_dev_to_conn(dev); struct device *parent = conn->dev.parent; + kfree(conn->persistent_address); kfree(conn); put_device(parent); } @@ -301,12 +406,16 @@ static int iscsi_is_conn_dev(const struc * This can be called from a LLD or iscsi_transport. The connection * is child of the session so cid must be unique for all connections * on the session. + * + * Since we do not support MCS, cid will normally be zero. In some cases + * for software iscsi we could be trying to preallocate a connection struct + * in which case there could be two connection structs and cid would be + * non-zero. **/ struct iscsi_cls_conn * iscsi_create_conn(struct iscsi_cls_session *session, uint32_t cid) { struct iscsi_transport *transport = session->transport; - struct Scsi_Host *shost = iscsi_session_to_shost(session); struct iscsi_cls_conn *conn; int err; @@ -319,12 +428,14 @@ iscsi_create_conn(struct iscsi_cls_sessi INIT_LIST_HEAD(&conn->conn_list); conn->transport = transport; + conn->cid = cid; /* this is released in the dev's release function */ if (!get_device(&session->dev)) goto free_conn; + snprintf(conn->dev.bus_id, BUS_ID_SIZE, "connection%d:%u", - shost->host_no, cid); + session->sid, cid); conn->dev.parent = &session->dev; conn->dev.release = iscsi_conn_release; err = device_register(&conn->dev); @@ -361,105 +472,6 @@ int iscsi_destroy_conn(struct iscsi_cls_ EXPORT_SYMBOL_GPL(iscsi_destroy_conn); /* - * These functions are used only by software iscsi_transports - * which do not allocate and more their scsi_hosts since this - * is initiated from userspace. - */ - -/* - * iSCSI Session's hostdata organization: - * - * *------------------* <== hostdata_session(host->hostdata) - * | ptr to class sess| - * |------------------| <== iscsi_hostdata(host->hostdata) - * | transport's data | - * *------------------* - */ - -#define hostdata_privsize(_t) (sizeof(unsigned long) + _t->hostdata_size + \ - _t->hostdata_size % sizeof(unsigned long)) - -#define hostdata_session(_hostdata) (iscsi_ptr(*(unsigned long *)_hostdata)) - -/** - * iscsi_transport_create_session - create iscsi cls session and host - * scsit: scsi transport template - * transport: iscsi transport template - * - * This can be used by software iscsi_transports that allocate - * a session per scsi host. - **/ -struct Scsi_Host * -iscsi_transport_create_session(struct scsi_transport_template *scsit, - struct iscsi_transport *transport) -{ - struct iscsi_cls_session *session; - struct Scsi_Host *shost; - unsigned long flags; - - shost = scsi_host_alloc(transport->host_template, - hostdata_privsize(transport)); - if (!shost) { - printk(KERN_ERR "iscsi: can not allocate SCSI host for " - "session\n"); - return NULL; - } - - shost->max_id = 1; - shost->max_channel = 0; - shost->max_lun = transport->max_lun; - shost->max_cmd_len = transport->max_cmd_len; - shost->transportt = scsit; - shost->transportt->create_work_queue = 1; - - if (scsi_add_host(shost, NULL)) - goto free_host; - - session = iscsi_create_session(shost, transport); - if (!session) - goto remove_host; - - *(unsigned long*)shost->hostdata = (unsigned long)session; - spin_lock_irqsave(&sesslock, flags); - list_add(&session->sess_list, &sesslist); - spin_unlock_irqrestore(&sesslock, flags); - return shost; - -remove_host: - scsi_remove_host(shost); -free_host: - scsi_host_put(shost); - return NULL; -} - -EXPORT_SYMBOL_GPL(iscsi_transport_create_session); - -/** - * iscsi_transport_destroy_session - destroy session and scsi host - * shost: scsi host - * - * This can be used by software iscsi_transports that allocate - * a session per scsi host. - **/ -int iscsi_transport_destroy_session(struct Scsi_Host *shost) -{ - struct iscsi_cls_session *session; - unsigned long flags; - - scsi_remove_host(shost); - session = hostdata_session(shost->hostdata); - spin_lock_irqsave(&sesslock, flags); - list_del(&session->sess_list); - spin_unlock_irqrestore(&sesslock, flags); - iscsi_destroy_session(session); - /* ref from host alloc */ - scsi_host_put(shost); - return 0; -} - -EXPORT_SYMBOL_GPL(iscsi_transport_destroy_session); - -/* * iscsi interface functions */ static struct iscsi_internal * @@ -574,6 +586,7 @@ iscsi_unicast_skb(struct mempool_zone *z } spin_lock_irqsave(&zone->freelock, flags); + INIT_LIST_HEAD(skb_to_lh(skb)); list_add(skb_to_lh(skb), &zone->freequeue); spin_unlock_irqrestore(&zone->freelock, flags); @@ -607,7 +620,8 @@ int iscsi_recv_pdu(struct iscsi_cls_conn ev->type = ISCSI_KEVENT_RECV_PDU; if (atomic_read(&conn->z_pdu->allocated) >= conn->z_pdu->hiwat) ev->iferror = -ENOMEM; - ev->r.recv_req.conn_handle = iscsi_handle(conn); + ev->r.recv_req.cid = conn->cid; + ev->r.recv_req.sid = iscsi_conn_get_sid(conn); pdu = (char*)ev + sizeof(*ev); memcpy(pdu, hdr, sizeof(struct iscsi_hdr)); memcpy(pdu + sizeof(struct iscsi_hdr), data, data_size); @@ -639,7 +653,8 @@ void iscsi_conn_error(struct iscsi_cls_c if (atomic_read(&conn->z_error->allocated) >= conn->z_error->hiwat) ev->iferror = -ENOMEM; ev->r.connerror.error = error; - ev->r.connerror.conn_handle = iscsi_handle(conn); + ev->r.connerror.cid = conn->cid; + ev->r.connerror.sid = iscsi_conn_get_sid(conn); iscsi_unicast_skb(conn->z_error, skb); @@ -689,7 +704,7 @@ iscsi_if_get_stats(struct iscsi_transpor ISCSI_STATS_CUSTOM_MAX); int err = 0; - conn = iscsi_conn_lookup(ev->u.get_stats.conn_handle); + conn = iscsi_conn_lookup(ev->u.get_stats.sid, ev->u.get_stats.cid); if (!conn) return -EEXIST; @@ -713,8 +728,10 @@ iscsi_if_get_stats(struct iscsi_transpor evstat->type = nlh->nlmsg_type; if (atomic_read(&conn->z_pdu->allocated) >= conn->z_pdu->hiwat) evstat->iferror = -ENOMEM; - evstat->u.get_stats.conn_handle = - ev->u.get_stats.conn_handle; + evstat->u.get_stats.cid = + ev->u.get_stats.cid; + evstat->u.get_stats.sid = + ev->u.get_stats.sid; stats = (struct iscsi_stats *) ((char*)evstat + sizeof(*evstat)); memset(stats, 0, sizeof(*stats)); @@ -740,16 +757,21 @@ iscsi_if_create_session(struct iscsi_int { struct iscsi_transport *transport = priv->iscsi_transport; struct iscsi_cls_session *session; - uint32_t sid; + unsigned long flags; + uint32_t hostno; - session = transport->create_session(&priv->t, + session = transport->create_session(transport, &priv->t, ev->u.c_session.initial_cmdsn, - &sid); + &hostno); if (!session) return -ENOMEM; - ev->r.c_session_ret.session_handle = iscsi_handle(session); - ev->r.c_session_ret.sid = sid; + spin_lock_irqsave(&sesslock, flags); + list_add(&session->sess_list, &sesslist); + spin_unlock_irqrestore(&sesslock, flags); + + ev->r.c_session_ret.host_no = hostno; + ev->r.c_session_ret.sid = session->sid; return 0; } @@ -760,13 +782,20 @@ iscsi_if_create_conn(struct iscsi_transp struct iscsi_cls_session *session; unsigned long flags; - session = iscsi_session_lookup(ev->u.c_conn.session_handle); - if (!session) + session = iscsi_session_lookup(ev->u.c_conn.sid); + if (!session) { + printk(KERN_ERR "iscsi: invalid session %d\n", + ev->u.c_conn.sid); return -EINVAL; + } conn = transport->create_conn(session, ev->u.c_conn.cid); - if (!conn) + if (!conn) { + printk(KERN_ERR "iscsi: couldn't create a new " + "connection for session %d\n", + session->sid); return -ENOMEM; + } conn->z_pdu = mempool_zone_init(Z_MAX_PDU, NLMSG_SPACE(sizeof(struct iscsi_uevent) + @@ -788,7 +817,8 @@ iscsi_if_create_conn(struct iscsi_transp goto free_pdu_pool; } - ev->r.handle = iscsi_handle(conn); + ev->r.c_conn_ret.sid = session->sid; + ev->r.c_conn_ret.cid = conn->cid; spin_lock_irqsave(&connlock, flags); list_add(&conn->conn_list, &connlist); @@ -812,7 +842,7 @@ iscsi_if_destroy_conn(struct iscsi_trans struct iscsi_cls_conn *conn; struct mempool_zone *z_error, *z_pdu; - conn = iscsi_conn_lookup(ev->u.d_conn.conn_handle); + conn = iscsi_conn_lookup(ev->u.d_conn.sid, ev->u.d_conn.cid); if (!conn) return -EINVAL; spin_lock_irqsave(&connlock, flags); @@ -832,6 +862,72 @@ iscsi_if_destroy_conn(struct iscsi_trans return 0; } +static void +iscsi_copy_param(struct iscsi_uevent *ev, uint32_t *value, char *data) +{ + if (ev->u.set_param.len != sizeof(uint32_t)) + BUG(); + memcpy(value, data, min_t(uint32_t, sizeof(uint32_t), + ev->u.set_param.len)); +} + +static int +iscsi_set_param(struct iscsi_transport *transport, struct iscsi_uevent *ev) +{ + char *data = (char*)ev + sizeof(*ev); + struct iscsi_cls_conn *conn; + struct iscsi_cls_session *session; + int err = 0; + uint32_t value = 0; + + session = iscsi_session_lookup(ev->u.set_param.sid); + conn = iscsi_conn_lookup(ev->u.set_param.sid, ev->u.set_param.cid); + if (!conn || !session) + return -EINVAL; + + switch (ev->u.set_param.param) { + case ISCSI_PARAM_SESS_RECOVERY_TMO: + iscsi_copy_param(ev, &value, data); + if (value != 0) + session->recovery_tmo = value; + break; + case ISCSI_PARAM_TARGET_NAME: + /* this should not change between logins */ + if (session->targetname) + return 0; + + session->targetname = kstrdup(data, GFP_KERNEL); + if (!session->targetname) + return -ENOMEM; + break; + case ISCSI_PARAM_TPGT: + iscsi_copy_param(ev, &value, data); + session->tpgt = value; + break; + case ISCSI_PARAM_PERSISTENT_PORT: + iscsi_copy_param(ev, &value, data); + conn->persistent_port = value; + break; + case ISCSI_PARAM_PERSISTENT_ADDRESS: + /* + * this is the address returned in discovery so it should + * not change between logins. + */ + if (conn->persistent_address) + return 0; + + conn->persistent_address = kstrdup(data, GFP_KERNEL); + if (!conn->persistent_address) + return -ENOMEM; + break; + default: + iscsi_copy_param(ev, &value, data); + err = transport->set_param(conn, ev->u.set_param.param, value); + } + + return err; +} + static int iscsi_if_recv_msg(struct sk_buff *skb, struct nlmsghdr *nlh) { @@ -841,6 +937,7 @@ iscsi_if_recv_msg(struct sk_buff *skb, s struct iscsi_internal *priv; struct iscsi_cls_session *session; struct iscsi_cls_conn *conn; + unsigned long flags; priv = iscsi_if_transport_lookup(iscsi_ptr(ev->transport_handle)); if (!priv) @@ -855,10 +952,14 @@ iscsi_if_recv_msg(struct sk_buff *skb, s err = iscsi_if_create_session(priv, ev); break; case ISCSI_UEVENT_DESTROY_SESSION: - session = iscsi_session_lookup(ev->u.d_session.session_handle); - if (session) + session = iscsi_session_lookup(ev->u.d_session.sid); + if (session) { + spin_lock_irqsave(&sesslock, flags); + list_del(&session->sess_list); + spin_unlock_irqrestore(&sesslock, flags); + transport->destroy_session(session); - else + } else err = -EINVAL; break; case ISCSI_UEVENT_CREATE_CONN: @@ -868,8 +969,8 @@ iscsi_if_recv_msg(struct sk_buff *skb, s err = iscsi_if_destroy_conn(transport, ev); break; case ISCSI_UEVENT_BIND_CONN: - session = iscsi_session_lookup(ev->u.b_conn.session_handle); - conn = iscsi_conn_lookup(ev->u.b_conn.conn_handle); + session = iscsi_session_lookup(ev->u.b_conn.sid); + conn = iscsi_conn_lookup(ev->u.b_conn.sid, ev->u.b_conn.cid); if (session && conn) ev->r.retcode = transport->bind_conn(session, conn, @@ -879,30 +980,24 @@ iscsi_if_recv_msg(struct sk_buff *skb, s err = -EINVAL; break; case ISCSI_UEVENT_SET_PARAM: - conn = iscsi_conn_lookup(ev->u.set_param.conn_handle); - if (conn) - ev->r.retcode = transport->set_param(conn, - ev->u.set_param.param, ev->u.set_param.value); - else - err = -EINVAL; + err = iscsi_set_param(transport, ev); break; case ISCSI_UEVENT_START_CONN: - conn = iscsi_conn_lookup(ev->u.start_conn.conn_handle); + conn = iscsi_conn_lookup(ev->u.start_conn.sid, ev->u.start_conn.cid); if (conn) ev->r.retcode = transport->start_conn(conn); else err = -EINVAL; - break; case ISCSI_UEVENT_STOP_CONN: - conn = iscsi_conn_lookup(ev->u.stop_conn.conn_handle); + conn = iscsi_conn_lookup(ev->u.stop_conn.sid, ev->u.stop_conn.cid); if (conn) transport->stop_conn(conn, ev->u.stop_conn.flag); else err = -EINVAL; break; case ISCSI_UEVENT_SEND_PDU: - conn = iscsi_conn_lookup(ev->u.send_pdu.conn_handle); + conn = iscsi_conn_lookup(ev->u.send_pdu.sid, ev->u.send_pdu.cid); if (conn) ev->r.retcode = transport->send_pdu(conn, (struct iscsi_hdr*)((char*)ev + sizeof(*ev)), @@ -923,9 +1018,11 @@ iscsi_if_recv_msg(struct sk_buff *skb, s return err; } -/* Get message from skb (based on rtnetlink_rcv_skb). Each message is - * processed by iscsi_if_recv_msg. Malformed skbs with wrong length are - * or invalid creds discarded silently. */ +/* + * Get message from skb (based on rtnetlink_rcv_skb). Each message is + * processed by iscsi_if_recv_msg. Malformed skbs with wrong lengths or + * invalid creds are discarded silently. + */ static void iscsi_if_rx(struct sock *sk, int len) { @@ -988,6 +1085,10 @@ free_skb: #define iscsi_cdev_to_conn(_cdev) \ iscsi_dev_to_conn(_cdev->dev) +#define ISCSI_CLASS_ATTR(_prefix,_name,_mode,_show,_store) \ +struct class_device_attribute class_device_attr_##_prefix##_##_name = \ + __ATTR(_name,_mode,_show,_store) + /* * iSCSI connection attrs */ @@ -1005,7 +1106,8 @@ show_conn_int_param_##param(struct class #define iscsi_conn_int_attr(field, param, format) \ iscsi_conn_int_attr_show(param, format) \ -static CLASS_DEVICE_ATTR(field, S_IRUGO, show_conn_int_param_##param, NULL); +static ISCSI_CLASS_ATTR(conn, field, S_IRUGO, show_conn_int_param_##param, \ + NULL); iscsi_conn_int_attr(max_recv_dlength, ISCSI_PARAM_MAX_RECV_DLENGTH, "%u"); iscsi_conn_int_attr(max_xmit_dlength, ISCSI_PARAM_MAX_XMIT_DLENGTH, "%u"); @@ -1013,6 +1115,25 @@ iscsi_conn_int_attr(header_digest, ISCSI iscsi_conn_int_attr(data_digest, ISCSI_PARAM_DATADGST_EN, "%d"); iscsi_conn_int_attr(ifmarker, ISCSI_PARAM_IFMARKER_EN, "%d"); iscsi_conn_int_attr(ofmarker, ISCSI_PARAM_OFMARKER_EN, "%d"); +iscsi_conn_int_attr(persistent_port, ISCSI_PARAM_PERSISTENT_PORT, "%d"); +iscsi_conn_int_attr(port, ISCSI_PARAM_CONN_PORT, "%d"); + +#define iscsi_conn_str_attr_show(param) \ +static ssize_t \ +show_conn_str_param_##param(struct class_device *cdev, char *buf) \ +{ \ + struct iscsi_cls_conn *conn = iscsi_cdev_to_conn(cdev); \ + struct iscsi_transport *t = conn->transport; \ + return t->get_conn_str_param(conn, param, buf); \ +} + +#define iscsi_conn_str_attr(field, param) \ + iscsi_conn_str_attr_show(param) \ +static ISCSI_CLASS_ATTR(conn, field, S_IRUGO, show_conn_str_param_##param, \ + NULL); + +iscsi_conn_str_attr(persistent_address, ISCSI_PARAM_PERSISTENT_ADDRESS); +iscsi_conn_str_attr(address, ISCSI_PARAM_CONN_ADDRESS); #define iscsi_cdev_to_session(_cdev) \ iscsi_dev_to_session(_cdev->dev) @@ -1034,7 +1155,8 @@ show_session_int_param_##param(struct cl #define iscsi_session_int_attr(field, param, format) \ iscsi_session_int_attr_show(param, format) \ -static CLASS_DEVICE_ATTR(field, S_IRUGO, show_session_int_param_##param, NULL); +static ISCSI_CLASS_ATTR(sess, field, S_IRUGO, show_session_int_param_##param, \ + NULL); iscsi_session_int_attr(initial_r2t, ISCSI_PARAM_INITIAL_R2T_EN, "%d"); iscsi_session_int_attr(max_outstanding_r2t, ISCSI_PARAM_MAX_R2T, "%hu"); @@ -1044,18 +1166,89 @@ iscsi_session_int_attr(max_burst_len, IS iscsi_session_int_attr(data_pdu_in_order, ISCSI_PARAM_PDU_INORDER_EN, "%d"); iscsi_session_int_attr(data_seq_in_order, ISCSI_PARAM_DATASEQ_INORDER_EN, "%d"); iscsi_session_int_attr(erl, ISCSI_PARAM_ERL, "%d"); +iscsi_session_int_attr(tpgt, ISCSI_PARAM_TPGT, "%d"); + +#define iscsi_session_str_attr_show(param) \ +static ssize_t \ +show_session_str_param_##param(struct class_device *cdev, char *buf) \ +{ \ + struct iscsi_cls_session *session = iscsi_cdev_to_session(cdev); \ + struct iscsi_transport *t = session->transport; \ + return t->get_session_str_param(session, param, buf); \ +} + +#define iscsi_session_str_attr(field, param) \ + iscsi_session_str_attr_show(param) \ +static ISCSI_CLASS_ATTR(sess, field, S_IRUGO, show_session_str_param_##param, \ + NULL); + +iscsi_session_str_attr(targetname, ISCSI_PARAM_TARGET_NAME); + +/* + * Private session and conn attrs. userspace uses several iscsi values + * to identify each session between reboots. Some of these values may not + * be present in the iscsi_transport/LLD driver becuase userspace handles + * login (and failback for login redirect) so for these type of drivers + * the class manages the attrs and values for the iscsi_transport/LLD + */ +#define iscsi_priv_session_attr_show(field, format) \ +static ssize_t \ +show_priv_session_##field(struct class_device *cdev, char *buf) \ +{ \ + struct iscsi_cls_session *session = iscsi_cdev_to_session(cdev); \ + return sprintf(buf, format"\n", session->field); \ +} + +#define iscsi_priv_session_attr(field, format) \ + iscsi_priv_session_attr_show(field, format) \ +static ISCSI_CLASS_ATTR(priv_sess, field, S_IRUGO, show_priv_session_##field, \ + NULL) +iscsi_priv_session_attr(targetname, "%s"); +iscsi_priv_session_attr(tpgt, "%d"); +iscsi_priv_session_attr(recovery_tmo, "%d"); + +#define iscsi_priv_conn_attr_show(field, format) \ +static ssize_t \ +show_priv_conn_##field(struct class_device *cdev, char *buf) \ +{ \ + struct iscsi_cls_conn *conn = iscsi_cdev_to_conn(cdev); \ + return sprintf(buf, format"\n", conn->field); \ +} -#define SETUP_SESSION_RD_ATTR(field, param) \ - if (priv->param_mask & (1 << param)) { \ - priv->session_attrs[count] = &class_device_attr_##field;\ +#define iscsi_priv_conn_attr(field, format) \ + iscsi_priv_conn_attr_show(field, format) \ +static ISCSI_CLASS_ATTR(priv_conn, field, S_IRUGO, show_priv_conn_##field, \ + NULL) +iscsi_priv_conn_attr(persistent_address, "%s"); +iscsi_priv_conn_attr(persistent_port, "%d"); + +#define SETUP_PRIV_SESSION_RD_ATTR(field) \ +do { \ + priv->session_attrs[count] = &class_device_attr_priv_sess_##field; \ + count++; \ +} while (0) + +#define SETUP_SESSION_RD_ATTR(field, param_flag) \ +do { \ + if (tt->param_mask & param_flag) { \ + priv->session_attrs[count] = &class_device_attr_sess_##field; \ count++; \ - } + } \ +} while (0) -#define SETUP_CONN_RD_ATTR(field, param) \ - if (priv->param_mask & (1 << param)) { \ - priv->conn_attrs[count] = &class_device_attr_##field; \ +#define SETUP_PRIV_CONN_RD_ATTR(field) \ +do { \ + priv->conn_attrs[count] = &class_device_attr_priv_conn_##field; \ + count++; \ +} while (0) + +#define SETUP_CONN_RD_ATTR(field, param_flag) \ +do { \ + if (tt->param_mask & param_flag) { \ + priv->conn_attrs[count] = &class_device_attr_conn_##field; \ count++; \ - } + } \ +} while (0) static int iscsi_session_match(struct attribute_container *cont, struct device *dev) @@ -1104,6 +1297,24 @@ static int iscsi_conn_match(struct attri return &priv->conn_cont.ac == cont; } +static int iscsi_host_match(struct attribute_container *cont, + struct device *dev) +{ + struct Scsi_Host *shost; + struct iscsi_internal *priv; + + if (!scsi_is_host_device(dev)) + return 0; + + shost = dev_to_shost(dev); + if (!shost->transportt || + shost->transportt->host_attrs.ac.class != &iscsi_host_class.class) + return 0; + + priv = to_iscsi_internal(shost->transportt); + return &priv->t.host_attrs.ac == cont; +} + struct scsi_transport_template * iscsi_register_transport(struct iscsi_transport *tt) { @@ -1122,6 +1333,7 @@ iscsi_register_transport(struct iscsi_tr return NULL; INIT_LIST_HEAD(&priv->list); priv->iscsi_transport = tt; + priv->t.user_scan = iscsi_user_scan; priv->cdev.class = &iscsi_transport_class; snprintf(priv->cdev.class_id, BUS_ID_SIZE, "%s", tt->name); @@ -1133,18 +1345,13 @@ iscsi_register_transport(struct iscsi_tr if (err) goto unregister_cdev; - /* setup parameters mask */ - priv->param_mask = 0xFFFFFFFF; - if (!(tt->caps & CAP_MULTI_R2T)) - priv->param_mask &= ~(1 << ISCSI_PARAM_MAX_R2T); - if (!(tt->caps & CAP_HDRDGST)) - priv->param_mask &= ~(1 << ISCSI_PARAM_HDRDGST_EN); - if (!(tt->caps & CAP_DATADGST)) - priv->param_mask &= ~(1 << ISCSI_PARAM_DATADGST_EN); - if (!(tt->caps & CAP_MARKERS)) { - priv->param_mask &= ~(1 << ISCSI_PARAM_IFMARKER_EN); - priv->param_mask &= ~(1 << ISCSI_PARAM_OFMARKER_EN); - } + /* host parameters */ + priv->t.host_attrs.ac.attrs = &priv->host_attrs[0]; + priv->t.host_attrs.ac.class = &iscsi_host_class.class; + priv->t.host_attrs.ac.match = iscsi_host_match; + priv->t.host_size = sizeof(struct iscsi_host); + priv->host_attrs[0] = NULL; + transport_container_register(&priv->t.host_attrs); /* connection parameters */ priv->conn_cont.ac.attrs = &priv->conn_attrs[0]; @@ -1152,12 +1359,24 @@ iscsi_register_transport(struct iscsi_tr priv->conn_cont.ac.match = iscsi_conn_match; transport_container_register(&priv->conn_cont); - SETUP_CONN_RD_ATTR(max_recv_dlength, ISCSI_PARAM_MAX_RECV_DLENGTH); - SETUP_CONN_RD_ATTR(max_xmit_dlength, ISCSI_PARAM_MAX_XMIT_DLENGTH); - SETUP_CONN_RD_ATTR(header_digest, ISCSI_PARAM_HDRDGST_EN); - SETUP_CONN_RD_ATTR(data_digest, ISCSI_PARAM_DATADGST_EN); - SETUP_CONN_RD_ATTR(ifmarker, ISCSI_PARAM_IFMARKER_EN); - SETUP_CONN_RD_ATTR(ofmarker, ISCSI_PARAM_OFMARKER_EN); + SETUP_CONN_RD_ATTR(max_recv_dlength, ISCSI_MAX_RECV_DLENGTH); + SETUP_CONN_RD_ATTR(max_xmit_dlength, ISCSI_MAX_XMIT_DLENGTH); + SETUP_CONN_RD_ATTR(header_digest, ISCSI_HDRDGST_EN); + SETUP_CONN_RD_ATTR(data_digest, ISCSI_DATADGST_EN); + SETUP_CONN_RD_ATTR(ifmarker, ISCSI_IFMARKER_EN); + SETUP_CONN_RD_ATTR(ofmarker, ISCSI_OFMARKER_EN); + SETUP_CONN_RD_ATTR(address, ISCSI_CONN_ADDRESS); + SETUP_CONN_RD_ATTR(port, ISCSI_CONN_PORT); + + if (tt->param_mask & ISCSI_PERSISTENT_ADDRESS) + SETUP_CONN_RD_ATTR(persistent_address, ISCSI_PERSISTENT_ADDRESS); + else + SETUP_PRIV_CONN_RD_ATTR(persistent_address); + + if (tt->param_mask & ISCSI_PERSISTENT_PORT) + SETUP_CONN_RD_ATTR(persistent_port, ISCSI_PERSISTENT_PORT); + else + SETUP_PRIV_CONN_RD_ATTR(persistent_port); BUG_ON(count > ISCSI_CONN_ATTRS); priv->conn_attrs[count] = NULL; @@ -1169,14 +1388,25 @@ iscsi_register_transport(struct iscsi_tr priv->session_cont.ac.match = iscsi_session_match; transport_container_register(&priv->session_cont); - SETUP_SESSION_RD_ATTR(initial_r2t, ISCSI_PARAM_INITIAL_R2T_EN); - SETUP_SESSION_RD_ATTR(max_outstanding_r2t, ISCSI_PARAM_MAX_R2T); - SETUP_SESSION_RD_ATTR(immediate_data, ISCSI_PARAM_IMM_DATA_EN); - SETUP_SESSION_RD_ATTR(first_burst_len, ISCSI_PARAM_FIRST_BURST); - SETUP_SESSION_RD_ATTR(max_burst_len, ISCSI_PARAM_MAX_BURST); - SETUP_SESSION_RD_ATTR(data_pdu_in_order, ISCSI_PARAM_PDU_INORDER_EN); - SETUP_SESSION_RD_ATTR(data_seq_in_order,ISCSI_PARAM_DATASEQ_INORDER_EN) - SETUP_SESSION_RD_ATTR(erl, ISCSI_PARAM_ERL); + SETUP_SESSION_RD_ATTR(initial_r2t, ISCSI_INITIAL_R2T_EN); + SETUP_SESSION_RD_ATTR(max_outstanding_r2t, ISCSI_MAX_R2T); + SETUP_SESSION_RD_ATTR(immediate_data, ISCSI_IMM_DATA_EN); + SETUP_SESSION_RD_ATTR(first_burst_len, ISCSI_FIRST_BURST); + SETUP_SESSION_RD_ATTR(max_burst_len, ISCSI_MAX_BURST); + SETUP_SESSION_RD_ATTR(data_pdu_in_order, ISCSI_PDU_INORDER_EN); + SETUP_SESSION_RD_ATTR(data_seq_in_order, ISCSI_DATASEQ_INORDER_EN); + SETUP_SESSION_RD_ATTR(erl, ISCSI_ERL); + SETUP_PRIV_SESSION_RD_ATTR(recovery_tmo); + + if (tt->param_mask & ISCSI_TARGET_NAME) + SETUP_SESSION_RD_ATTR(targetname, ISCSI_TARGET_NAME); + else + SETUP_PRIV_SESSION_RD_ATTR(targetname); + + if (tt->param_mask & ISCSI_TPGT) + SETUP_SESSION_RD_ATTR(tpgt, ISCSI_TPGT); + else + SETUP_PRIV_SESSION_RD_ATTR(tpgt); BUG_ON(count > ISCSI_SESSION_ATTRS); priv->session_attrs[count] = NULL; @@ -1214,6 +1444,7 @@ int iscsi_unregister_transport(struct is transport_container_unregister(&priv->conn_cont); transport_container_unregister(&priv->session_cont); + transport_container_unregister(&priv->t.host_attrs); sysfs_remove_group(&priv->cdev.kobj, &iscsi_transport_group); class_device_unregister(&priv->cdev); @@ -1257,10 +1488,14 @@ static __init int iscsi_transport_init(v if (err) return err; - err = transport_class_register(&iscsi_connection_class); + err = transport_class_register(&iscsi_host_class); if (err) goto unregister_transport_class; + err = transport_class_register(&iscsi_connection_class); + if (err) + goto unregister_host_class; + err = transport_class_register(&iscsi_session_class); if (err) goto unregister_conn_class; @@ -1288,6 +1523,8 @@ unregister_session_class: transport_class_unregister(&iscsi_session_class); unregister_conn_class: transport_class_unregister(&iscsi_connection_class); +unregister_host_class: + transport_class_unregister(&iscsi_host_class); unregister_transport_class: class_unregister(&iscsi_transport_class); return err; @@ -1300,6 +1537,7 @@ static void __exit iscsi_transport_exit( netlink_unregister_notifier(&iscsi_nl_notifier); transport_class_unregister(&iscsi_connection_class); transport_class_unregister(&iscsi_session_class); + transport_class_unregister(&iscsi_host_class); class_unregister(&iscsi_transport_class); } diff -puN include/scsi/iscsi_if.h~git-scsi-misc include/scsi/iscsi_if.h --- devel/include/scsi/iscsi_if.h~git-scsi-misc 2006-04-30 00:30:24.000000000 -0700 +++ devel-akpm/include/scsi/iscsi_if.h 2006-04-30 00:30:24.000000000 -0700 @@ -60,59 +60,68 @@ struct iscsi_uevent { uint32_t initial_cmdsn; } c_session; struct msg_destroy_session { - uint64_t session_handle; uint32_t sid; } d_session; struct msg_create_conn { - uint64_t session_handle; - uint32_t cid; uint32_t sid; + uint32_t cid; } c_conn; struct msg_bind_conn { - uint64_t session_handle; - uint64_t conn_handle; + uint32_t sid; + uint32_t cid; uint32_t transport_fd; uint32_t is_leading; } b_conn; struct msg_destroy_conn { - uint64_t conn_handle; + uint32_t sid; uint32_t cid; } d_conn; struct msg_send_pdu { + uint32_t sid; + uint32_t cid; uint32_t hdr_size; uint32_t data_size; - uint64_t conn_handle; } send_pdu; struct msg_set_param { - uint64_t conn_handle; + uint32_t sid; + uint32_t cid; uint32_t param; /* enum iscsi_param */ - uint32_t value; + uint32_t len; } set_param; struct msg_start_conn { - uint64_t conn_handle; + uint32_t sid; + uint32_t cid; } start_conn; struct msg_stop_conn { + uint32_t sid; + uint32_t cid; uint64_t conn_handle; uint32_t flag; } stop_conn; struct msg_get_stats { - uint64_t conn_handle; + uint32_t sid; + uint32_t cid; } get_stats; } u; union { /* messages k -> u */ - uint64_t handle; int retcode; struct msg_create_session_ret { - uint64_t session_handle; uint32_t sid; + uint32_t host_no; } c_session_ret; + struct msg_create_conn_ret { + uint32_t sid; + uint32_t cid; + } c_conn_ret; struct msg_recv_req { + uint32_t sid; + uint32_t cid; uint64_t recv_handle; - uint64_t conn_handle; } recv_req; struct msg_conn_error { - uint64_t conn_handle; + uint32_t sid; + uint32_t cid; uint32_t error; /* enum iscsi_err */ } connerror; } r; @@ -139,29 +148,64 @@ enum iscsi_err { ISCSI_ERR_SESSION_FAILED = ISCSI_ERR_BASE + 13, ISCSI_ERR_HDR_DGST = ISCSI_ERR_BASE + 14, ISCSI_ERR_DATA_DGST = ISCSI_ERR_BASE + 15, - ISCSI_ERR_PARAM_NOT_FOUND = ISCSI_ERR_BASE + 16 + ISCSI_ERR_PARAM_NOT_FOUND = ISCSI_ERR_BASE + 16, + ISCSI_ERR_NO_SCSI_CMD = ISCSI_ERR_BASE + 17, }; /* * iSCSI Parameters (RFC3720) */ enum iscsi_param { - ISCSI_PARAM_MAX_RECV_DLENGTH = 0, - ISCSI_PARAM_MAX_XMIT_DLENGTH = 1, - ISCSI_PARAM_HDRDGST_EN = 2, - ISCSI_PARAM_DATADGST_EN = 3, - ISCSI_PARAM_INITIAL_R2T_EN = 4, - ISCSI_PARAM_MAX_R2T = 5, - ISCSI_PARAM_IMM_DATA_EN = 6, - ISCSI_PARAM_FIRST_BURST = 7, - ISCSI_PARAM_MAX_BURST = 8, - ISCSI_PARAM_PDU_INORDER_EN = 9, - ISCSI_PARAM_DATASEQ_INORDER_EN = 10, - ISCSI_PARAM_ERL = 11, - ISCSI_PARAM_IFMARKER_EN = 12, - ISCSI_PARAM_OFMARKER_EN = 13, + /* passed in using netlink set param */ + ISCSI_PARAM_MAX_RECV_DLENGTH, + ISCSI_PARAM_MAX_XMIT_DLENGTH, + ISCSI_PARAM_HDRDGST_EN, + ISCSI_PARAM_DATADGST_EN, + ISCSI_PARAM_INITIAL_R2T_EN, + ISCSI_PARAM_MAX_R2T, + ISCSI_PARAM_IMM_DATA_EN, + ISCSI_PARAM_FIRST_BURST, + ISCSI_PARAM_MAX_BURST, + ISCSI_PARAM_PDU_INORDER_EN, + ISCSI_PARAM_DATASEQ_INORDER_EN, + ISCSI_PARAM_ERL, + ISCSI_PARAM_IFMARKER_EN, + ISCSI_PARAM_OFMARKER_EN, + ISCSI_PARAM_TARGET_NAME, + ISCSI_PARAM_TPGT, + ISCSI_PARAM_PERSISTENT_ADDRESS, + ISCSI_PARAM_PERSISTENT_PORT, + ISCSI_PARAM_SESS_RECOVERY_TMO, + + /* pased in through bind conn using transport_fd */ + ISCSI_PARAM_CONN_PORT, + ISCSI_PARAM_CONN_ADDRESS, + + /* must always be last */ + ISCSI_PARAM_MAX, }; -#define ISCSI_PARAM_MAX 14 + +#define ISCSI_MAX_RECV_DLENGTH (1 << ISCSI_PARAM_MAX_RECV_DLENGTH) +#define ISCSI_MAX_XMIT_DLENGTH (1 << ISCSI_PARAM_MAX_XMIT_DLENGTH) +#define ISCSI_HDRDGST_EN (1 << ISCSI_PARAM_HDRDGST_EN) +#define ISCSI_DATADGST_EN (1 << ISCSI_PARAM_DATADGST_EN) +#define ISCSI_INITIAL_R2T_EN (1 << ISCSI_PARAM_INITIAL_R2T_EN) +#define ISCSI_MAX_R2T (1 << ISCSI_PARAM_MAX_R2T) +#define ISCSI_IMM_DATA_EN (1 << ISCSI_PARAM_IMM_DATA_EN) +#define ISCSI_FIRST_BURST (1 << ISCSI_PARAM_FIRST_BURST) +#define ISCSI_MAX_BURST (1 << ISCSI_PARAM_MAX_BURST) +#define ISCSI_PDU_INORDER_EN (1 << ISCSI_PARAM_PDU_INORDER_EN) +#define ISCSI_DATASEQ_INORDER_EN (1 << ISCSI_PARAM_DATASEQ_INORDER_EN) +#define ISCSI_ERL (1 << ISCSI_PARAM_ERL) +#define ISCSI_IFMARKER_EN (1 << ISCSI_PARAM_IFMARKER_EN) +#define ISCSI_OFMARKER_EN (1 << ISCSI_PARAM_OFMARKER_EN) +#define ISCSI_TARGET_NAME (1 << ISCSI_PARAM_TARGET_NAME) +#define ISCSI_TPGT (1 << ISCSI_PARAM_TPGT) +#define ISCSI_PERSISTENT_ADDRESS (1 << ISCSI_PARAM_PERSISTENT_ADDRESS) +#define ISCSI_PERSISTENT_PORT (1 << ISCSI_PARAM_PERSISTENT_PORT) +#define ISCSI_SESS_RECOVERY_TMO (1 << ISCSI_PARAM_SESS_RECOVERY_TMO) +#define ISCSI_CONN_PORT (1 << ISCSI_PARAM_CONN_PORT) +#define ISCSI_CONN_ADDRESS (1 << ISCSI_PARAM_CONN_ADDRESS) #define iscsi_ptr(_handle) ((void*)(unsigned long)_handle) #define iscsi_handle(_ptr) ((uint64_t)(unsigned long)_ptr) diff -puN /dev/null include/scsi/libiscsi.h --- /dev/null 2003-09-15 06:40:47.000000000 -0700 +++ devel-akpm/include/scsi/libiscsi.h 2006-04-30 00:30:24.000000000 -0700 @@ -0,0 +1,286 @@ +/* + * iSCSI lib definitions + * + * Copyright (C) 2006 Red Hat, Inc. All rights reserved. + * Copyright (C) 2004 - 2006 Mike Christie + * Copyright (C) 2004 - 2005 Dmitry Yusupov + * Copyright (C) 2004 - 2005 Alex Aizman + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + */ +#ifndef LIBISCSI_H +#define LIBISCSI_H + +#include +#include +#include +#include + +struct scsi_transport_template; +struct scsi_device; +struct Scsi_Host; +struct scsi_cmnd; +struct socket; +struct iscsi_transport; +struct iscsi_cls_session; +struct iscsi_cls_conn; +struct iscsi_session; +struct iscsi_nopin; + +/* #define DEBUG_SCSI */ +#ifdef DEBUG_SCSI +#define debug_scsi(fmt...) printk(KERN_INFO "scsi: " fmt) +#else +#define debug_scsi(fmt...) +#endif + +#define ISCSI_XMIT_CMDS_MAX 128 /* must be power of 2 */ +#define ISCSI_MGMT_CMDS_MAX 32 /* must be power of 2 */ +#define ISCSI_CONN_MAX 1 + +#define ISCSI_MGMT_ITT_OFFSET 0xa00 + +#define ISCSI_DEF_CMD_PER_LUN 32 +#define ISCSI_MAX_CMD_PER_LUN 128 + +/* Task Mgmt states */ +#define TMABORT_INITIAL 0x0 +#define TMABORT_SUCCESS 0x1 +#define TMABORT_FAILED 0x2 +#define TMABORT_TIMEDOUT 0x3 + +/* Connection suspend "bit" */ +#define ISCSI_SUSPEND_BIT 1 + +#define ISCSI_ITT_MASK (0xfff) +#define ISCSI_CID_SHIFT 12 +#define ISCSI_CID_MASK (0xffff << ISCSI_CID_SHIFT) +#define ISCSI_AGE_SHIFT 28 +#define ISCSI_AGE_MASK (0xf << ISCSI_AGE_SHIFT) + +struct iscsi_mgmt_task { + /* + * Becuae LLDs allocate their hdr differently, this is a pointer to + * that storage. It must be setup at session creation time. + */ + struct iscsi_hdr *hdr; + char *data; /* mgmt payload */ + int data_count; /* counts data to be sent */ + uint32_t itt; /* this ITT */ + void *dd_data; /* driver/transport data */ + struct list_head running; +}; + +struct iscsi_cmd_task { + /* + * Becuae LLDs allocate their hdr differently, this is a pointer to + * that storage. It must be setup at session creation time. + */ + struct iscsi_cmd *hdr; + int itt; /* this ITT */ + int datasn; /* DataSN */ + + uint32_t unsol_datasn; + int imm_count; /* imm-data (bytes) */ + int unsol_count; /* unsolicited (bytes)*/ + int data_count; /* remaining Data-Out */ + struct scsi_cmnd *sc; /* associated SCSI cmd*/ + int total_length; + struct iscsi_conn *conn; /* used connection */ + struct iscsi_mgmt_task *mtask; /* tmf mtask in progr */ + + struct list_head running; /* running cmd list */ + void *dd_data; /* driver/transport data */ +}; + +struct iscsi_conn { + struct iscsi_cls_conn *cls_conn; /* ptr to class connection */ + void *dd_data; /* iscsi_transport data */ + struct iscsi_session *session; /* parent session */ + /* + * LLDs should set this lock. It protects the transport recv + * code + */ + rwlock_t *recv_lock; + /* + * conn_stop() flag: stop to recover, stop to terminate + */ + int stop_stage; + + /* iSCSI connection-wide sequencing */ + uint32_t exp_statsn; + + /* control data */ + int id; /* CID */ + struct list_head item; /* maintains list of conns */ + int c_stage; /* connection state */ + struct iscsi_mgmt_task *login_mtask; /* mtask used for login/text */ + struct iscsi_mgmt_task *mtask; /* xmit mtask in progress */ + struct iscsi_cmd_task *ctask; /* xmit ctask in progress */ + + /* xmit */ + struct kfifo *immqueue; /* immediate xmit queue */ + struct kfifo *mgmtqueue; /* mgmt (control) xmit queue */ + struct list_head mgmt_run_list; /* list of control tasks */ + struct kfifo *xmitqueue; /* data-path cmd queue */ + struct list_head run_list; /* list of cmds in progress */ + struct work_struct xmitwork; /* per-conn. xmit workqueue */ + /* + * serializes connection xmit, access to kfifos: + * xmitqueue, immqueue, mgmtqueue + */ + struct mutex xmitmutex; + + unsigned long suspend_tx; /* suspend Tx */ + unsigned long suspend_rx; /* suspend Rx */ + + /* abort */ + wait_queue_head_t ehwait; /* used in eh_abort() */ + struct iscsi_tm tmhdr; + struct timer_list tmabort_timer; + int tmabort_state; /* see TMABORT_INITIAL, etc.*/ + + /* negotiated params */ + int max_recv_dlength; /* initiator_max_recv_dsl*/ + int max_xmit_dlength; /* target_max_recv_dsl */ + int hdrdgst_en; + int datadgst_en; + + /* MIB-statistics */ + uint64_t txdata_octets; + uint64_t rxdata_octets; + uint32_t scsicmd_pdus_cnt; + uint32_t dataout_pdus_cnt; + uint32_t scsirsp_pdus_cnt; + uint32_t datain_pdus_cnt; + uint32_t r2t_pdus_cnt; + uint32_t tmfcmd_pdus_cnt; + int32_t tmfrsp_pdus_cnt; + + /* custom statistics */ + uint32_t eh_abort_cnt; +}; + +struct iscsi_queue { + struct kfifo *queue; /* FIFO Queue */ + void **pool; /* Pool of elements */ + int max; /* Max number of elements */ +}; + +struct iscsi_session { + /* iSCSI session-wide sequencing */ + uint32_t cmdsn; + uint32_t exp_cmdsn; + uint32_t max_cmdsn; + + /* configuration */ + int initial_r2t_en; + int max_r2t; + int imm_data_en; + int first_burst; + int max_burst; + int time2wait; + int time2retain; + int pdu_inorder_en; + int dataseq_inorder_en; + int erl; + int ifmarker_en; + int ofmarker_en; + + /* control data */ + struct iscsi_transport *tt; + struct Scsi_Host *host; + struct iscsi_conn *leadconn; /* leading connection */ + spinlock_t lock; /* protects session state, * + * sequence numbers, * + * session resources: * + * - cmdpool, * + * - mgmtpool, * + * - r2tpool */ + int state; /* session state */ + int recovery_failed; + struct list_head item; + int conn_cnt; + int age; /* counts session re-opens */ + + struct list_head connections; /* list of connections */ + int cmds_max; /* size of cmds array */ + struct iscsi_cmd_task **cmds; /* Original Cmds arr */ + struct iscsi_queue cmdpool; /* PDU's pool */ + int mgmtpool_max; /* size of mgmt array */ + struct iscsi_mgmt_task **mgmt_cmds; /* Original mgmt arr */ + struct iscsi_queue mgmtpool; /* Mgmt PDU's pool */ +}; + +/* + * scsi host template + */ +extern int iscsi_change_queue_depth(struct scsi_device *sdev, int depth); +extern int iscsi_eh_abort(struct scsi_cmnd *sc); +extern int iscsi_eh_host_reset(struct scsi_cmnd *sc); +extern int iscsi_queuecommand(struct scsi_cmnd *sc, + void (*done)(struct scsi_cmnd *)); + +/* + * session management + */ +extern struct iscsi_cls_session * +iscsi_session_setup(struct iscsi_transport *, struct scsi_transport_template *, + int, int, uint32_t, uint32_t *); +extern void iscsi_session_teardown(struct iscsi_cls_session *); +extern struct iscsi_session *class_to_transport_session(struct iscsi_cls_session *); +extern void iscsi_start_session_recovery(struct iscsi_session *, + struct iscsi_conn *, int); +extern void iscsi_session_recovery_timedout(struct iscsi_cls_session *); + +#define session_to_cls(_sess) \ + hostdata_session(_sess->host->hostdata) + +/* + * connection management + */ +extern struct iscsi_cls_conn *iscsi_conn_setup(struct iscsi_cls_session *, + uint32_t); +extern void iscsi_conn_teardown(struct iscsi_cls_conn *); +extern int iscsi_conn_start(struct iscsi_cls_conn *); +extern void iscsi_conn_stop(struct iscsi_cls_conn *, int); +extern int iscsi_conn_bind(struct iscsi_cls_session *, struct iscsi_cls_conn *, + int); +extern void iscsi_conn_failure(struct iscsi_conn *conn, enum iscsi_err err); + +/* + * pdu and task processing + */ +extern int iscsi_check_assign_cmdsn(struct iscsi_session *, + struct iscsi_nopin *); +extern void iscsi_prep_unsolicit_data_pdu(struct iscsi_cmd_task *, + struct iscsi_data *hdr, + int transport_data_cnt); +extern int iscsi_conn_send_pdu(struct iscsi_cls_conn *, struct iscsi_hdr *, + char *, uint32_t); +extern int iscsi_complete_pdu(struct iscsi_conn *, struct iscsi_hdr *, + char *, int); +extern int __iscsi_complete_pdu(struct iscsi_conn *, struct iscsi_hdr *, + char *, int); +extern int iscsi_verify_itt(struct iscsi_conn *, struct iscsi_hdr *, + uint32_t *); + +/* + * generic helpers + */ +extern void iscsi_pool_free(struct iscsi_queue *, void **); +extern int iscsi_pool_init(struct iscsi_queue *, int, void ***, int); + +#endif diff -puN include/scsi/scsi_cmnd.h~git-scsi-misc include/scsi/scsi_cmnd.h --- devel/include/scsi/scsi_cmnd.h~git-scsi-misc 2006-04-30 00:30:24.000000000 -0700 +++ devel-akpm/include/scsi/scsi_cmnd.h 2006-04-30 00:30:24.000000000 -0700 @@ -152,4 +152,8 @@ extern void scsi_put_command(struct scsi extern void scsi_io_completion(struct scsi_cmnd *, unsigned int, unsigned int); extern void scsi_finish_command(struct scsi_cmnd *cmd); +extern void *scsi_kmap_atomic_sg(struct scatterlist *sg, int sg_count, + size_t *offset, size_t *len); +extern void scsi_kunmap_atomic_sg(void *virt); + #endif /* _SCSI_SCSI_CMND_H */ diff -puN include/scsi/scsi_devinfo.h~git-scsi-misc include/scsi/scsi_devinfo.h --- devel/include/scsi/scsi_devinfo.h~git-scsi-misc 2006-04-30 00:30:24.000000000 -0700 +++ devel-akpm/include/scsi/scsi_devinfo.h 2006-04-30 00:30:24.000000000 -0700 @@ -29,4 +29,5 @@ #define BLIST_SELECT_NO_ATN 0x200000 /* select without ATN */ #define BLIST_RETRY_HWERROR 0x400000 /* retry HARDWARE_ERROR */ #define BLIST_MAX_512 0x800000 /* maximum 512 sector cdb length */ +#define BLIST_ATTACH_PQ3 0x1000000 /* Scan: Attach to PQ3 devices */ #endif diff -puN include/scsi/scsi_transport_iscsi.h~git-scsi-misc include/scsi/scsi_transport_iscsi.h --- devel/include/scsi/scsi_transport_iscsi.h~git-scsi-misc 2006-04-30 00:30:24.000000000 -0700 +++ devel-akpm/include/scsi/scsi_transport_iscsi.h 2006-04-30 00:30:24.000000000 -0700 @@ -2,7 +2,7 @@ * iSCSI transport class definitions * * Copyright (C) IBM Corporation, 2004 - * Copyright (C) Mike Christie, 2004 - 2005 + * Copyright (C) Mike Christie, 2004 - 2006 * Copyright (C) Dmitry Yusupov, 2004 - 2005 * Copyright (C) Alex Aizman, 2004 - 2005 * @@ -27,9 +27,13 @@ #include struct scsi_transport_template; +struct iscsi_transport; struct Scsi_Host; struct mempool_zone; struct iscsi_cls_conn; +struct iscsi_conn; +struct iscsi_cmd_task; +struct iscsi_mgmt_task; /** * struct iscsi_transport - iSCSI Transport template @@ -46,6 +50,20 @@ struct iscsi_cls_conn; * @start_conn: set connection to be operational * @stop_conn: suspend/recover/terminate connection * @send_pdu: send iSCSI PDU, Login, Logout, NOP-Out, Reject, Text. + * @session_recovery_timedout: notify LLD a block during recovery timed out + * @suspend_conn_recv: susepend the recv side of the connection + * @termincate_conn: destroy socket connection. Called with mutex lock. + * @init_cmd_task: Initialize a iscsi_cmd_task and any internal structs. + * Called from queuecommand with session lock held. + * @init_mgmt_task: Initialize a iscsi_mgmt_task and any internal structs. + * Called from iscsi_conn_send_generic with xmitmutex. + * @xmit_cmd_task: requests LLD to transfer cmd task + * @xmit_mgmt_task: requests LLD to transfer mgmt task + * @cleanup_cmd_task: requests LLD to fail cmd task. Called with xmitmutex + * and session->lock after the connection has been + * suspended and terminated during recovery. If called + * from abort task then connection is not suspended + * or terminated but sk_callback_lock is held * * Template API provided by iSCSI Transport */ @@ -53,18 +71,18 @@ struct iscsi_transport { struct module *owner; char *name; unsigned int caps; + /* LLD sets this to indicate what values it can export to sysfs */ + unsigned int param_mask; struct scsi_host_template *host_template; - /* LLD session/scsi_host data size */ - int hostdata_size; - /* LLD iscsi_host data size */ - int ihostdata_size; /* LLD connection data size */ int conndata_size; + /* LLD session data size */ + int sessiondata_size; int max_lun; unsigned int max_conn; unsigned int max_cmd_len; - struct iscsi_cls_session *(*create_session) - (struct scsi_transport_template *t, uint32_t sn, uint32_t *sid); + struct iscsi_cls_session *(*create_session) (struct iscsi_transport *it, + struct scsi_transport_template *t, uint32_t sn, uint32_t *hn); void (*destroy_session) (struct iscsi_cls_session *session); struct iscsi_cls_conn *(*create_conn) (struct iscsi_cls_session *sess, uint32_t cid); @@ -77,14 +95,30 @@ struct iscsi_transport { int (*set_param) (struct iscsi_cls_conn *conn, enum iscsi_param param, uint32_t value); int (*get_conn_param) (struct iscsi_cls_conn *conn, - enum iscsi_param param, - uint32_t *value); + enum iscsi_param param, uint32_t *value); int (*get_session_param) (struct iscsi_cls_session *session, enum iscsi_param param, uint32_t *value); + int (*get_conn_str_param) (struct iscsi_cls_conn *conn, + enum iscsi_param param, char *buf); + int (*get_session_str_param) (struct iscsi_cls_session *session, + enum iscsi_param param, char *buf); int (*send_pdu) (struct iscsi_cls_conn *conn, struct iscsi_hdr *hdr, char *data, uint32_t data_size); void (*get_stats) (struct iscsi_cls_conn *conn, struct iscsi_stats *stats); + void (*suspend_conn_recv) (struct iscsi_conn *conn); + void (*terminate_conn) (struct iscsi_conn *conn); + void (*init_cmd_task) (struct iscsi_cmd_task *ctask); + void (*init_mgmt_task) (struct iscsi_conn *conn, + struct iscsi_mgmt_task *mtask, + char *data, uint32_t data_size); + int (*xmit_cmd_task) (struct iscsi_conn *conn, + struct iscsi_cmd_task *ctask); + void (*cleanup_cmd_task) (struct iscsi_conn *conn, + struct iscsi_cmd_task *ctask); + int (*xmit_mgmt_task) (struct iscsi_conn *conn, + struct iscsi_mgmt_task *mtask); + void (*session_recovery_timedout) (struct iscsi_cls_session *session); }; /* @@ -100,10 +134,26 @@ extern void iscsi_conn_error(struct iscs extern int iscsi_recv_pdu(struct iscsi_cls_conn *conn, struct iscsi_hdr *hdr, char *data, uint32_t data_size); + +/* Connection's states */ +#define ISCSI_CONN_INITIAL_STAGE 0 +#define ISCSI_CONN_STARTED 1 +#define ISCSI_CONN_STOPPED 2 +#define ISCSI_CONN_CLEANUP_WAIT 3 + struct iscsi_cls_conn { struct list_head conn_list; /* item in connlist */ void *dd_data; /* LLD private data */ struct iscsi_transport *transport; + uint32_t cid; /* connection id */ + + /* portal/group values we got during discovery */ + char *persistent_address; + int persistent_port; + /* portal/group values we are currently using */ + char *address; + int port; + int active; /* must be accessed with the connlock */ struct device dev; /* sysfs transport/container device */ struct mempool_zone *z_error; @@ -114,9 +164,30 @@ struct iscsi_cls_conn { #define iscsi_dev_to_conn(_dev) \ container_of(_dev, struct iscsi_cls_conn, dev) +/* Session's states */ +#define ISCSI_STATE_FREE 1 +#define ISCSI_STATE_LOGGED_IN 2 +#define ISCSI_STATE_FAILED 3 +#define ISCSI_STATE_TERMINATE 4 + struct iscsi_cls_session { struct list_head sess_list; /* item in session_list */ + struct list_head host_list; struct iscsi_transport *transport; + + /* iSCSI values used as unique id by userspace. */ + char *targetname; + int tpgt; + + /* recovery fields */ + int recovery_tmo; + struct work_struct recovery_work; + + int target_id; + int channel; + + int sid; /* session id */ + void *dd_data; /* LLD private data */ struct device dev; /* sysfs transport/container device */ }; @@ -126,22 +197,22 @@ struct iscsi_cls_session { #define iscsi_session_to_shost(_session) \ dev_to_shost(_session->dev.parent) +struct iscsi_host { + int next_target_id; + struct list_head sessions; + struct mutex mutex; +}; + /* * session and connection functions that can be used by HW iSCSI LLDs */ extern struct iscsi_cls_session *iscsi_create_session(struct Scsi_Host *shost, - struct iscsi_transport *t); + struct iscsi_transport *t, int channel); extern int iscsi_destroy_session(struct iscsi_cls_session *session); extern struct iscsi_cls_conn *iscsi_create_conn(struct iscsi_cls_session *sess, uint32_t cid); extern int iscsi_destroy_conn(struct iscsi_cls_conn *conn); - -/* - * session functions used by software iscsi - */ -extern struct Scsi_Host * -iscsi_transport_create_session(struct scsi_transport_template *scsit, - struct iscsi_transport *transport); -extern int iscsi_transport_destroy_session(struct Scsi_Host *shost); +extern void iscsi_unblock_session(struct iscsi_cls_session *session); +extern void iscsi_block_session(struct iscsi_cls_session *session); #endif _